TL;DR
Lucid-XR is a novel extended-reality data engine that generates diverse, realistic multi-modal data for training robotic systems, enabling zero-shot transfer in complex environments.
Contribution
It introduces a web-based physics simulation environment integrated with human-to-robot pose retargeting and a physics-guided video generation pipeline for robotic training data.
Findings
Zero-shot transfer of visual policies to unseen environments
Synthetic data enables effective training for complex manipulation tasks
System operates directly on XR headsets for immersive data collection
Abstract
We introduce Lucid-XR, a generative data engine for creating diverse and realistic-looking multi-modal data to train real-world robotic systems. At the core of Lucid-XR is vuer, a web-based physics simulation environment that runs directly on the XR headset, enabling internet-scale access to immersive, latency-free virtual interactions without requiring specialized equipment. The complete system integrates on-device physics simulation with human-to-robot pose retargeting. Data collected is further amplified by a physics-guided video generation pipeline steerable via natural language specifications. We demonstrate zero-shot transfer of robot visual policies to unseen, cluttered, and badly lit evaluation environments, after training entirely on Lucid-XR's synthetic data. We include examples across dexterous manipulation tasks that involve soft materials, loosely bound particles, and rigid…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
