RoboPanoptes: The All-seeing Robot with Whole-body Dexterity
Xiaomeng Xu, Dominik Bauer, Shuran Song

TL;DR
RoboPanoptes introduces a robot system with whole-body vision and dexterity, enabling complex manipulation tasks through a distributed camera system and visuomotor learning from demonstrations.
Contribution
It presents a novel robot architecture combining whole-body vision and dexterity, with a visuomotor policy learned from demonstrations for complex manipulation.
Findings
Successfully unboxes in narrow spaces
Performs multi-object sweeping and oversized object handling
Outperforms baselines in adaptability and efficiency
Abstract
We present RoboPanoptes, a capable yet practical robot system that achieves whole-body dexterity through whole-body vision. Its whole-body dexterity allows the robot to utilize its entire body surface for manipulation, such as leveraging multiple contact points or navigating constrained spaces. Meanwhile, whole-body vision uses a camera system distributed over the robot's surface to provide comprehensive, multi-perspective visual feedback of its own and the environment's state. At its core, RoboPanoptes uses a whole-body visuomotor policy that learns complex manipulation skills directly from human demonstrations, efficiently aggregating information from the distributed cameras while maintaining resilience to sensor failures. Together, these design aspects unlock new capabilities and tasks, allowing RoboPanoptes to unbox in narrow spaces, sweep multiple or oversized objects, and succeed…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsModular Robots and Swarm Intelligence · Micro and Nano Robotics · Neuroscience and Neural Engineering
