Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning

TL;DR

Bunny-VisionPro is a real-time, safe, and immersive bimanual teleoperation system using VR and low-cost haptic devices, significantly improving imitation learning for complex dexterous tasks.

Contribution

We introduce Bunny-VisionPro, a novel low-cost, real-time bimanual teleoperation system with enhanced safety and haptic feedback, enabling advanced imitation learning for complex manipulation tasks.

Findings

Outperforms prior systems on standard task suite with higher success rates

Reduces task completion times compared to existing teleoperation methods

Enables imitation learning for challenging multi-stage, long-horizon tasks

Abstract

Teleoperation is a crucial tool for collecting human demonstrations, but controlling robots with bimanual dexterous hands remains a challenge. Existing teleoperation systems struggle to handle the complexity of coordinating two hands for intricate manipulations. We introduce Bunny-VisionPro, a real-time bimanual dexterous teleoperation system that leverages a VR headset. Unlike previous vision-based teleoperation systems, we design novel low-cost devices to provide haptic feedback to the operator, enhancing immersion. Our system prioritizes safety by incorporating collision and singularity avoidance while maintaining real-time performance through innovative designs. Bunny-VisionPro outperforms prior systems on a standard task suite, achieving higher success rates and reduced task completion times. Moreover, the high-quality teleoperation demonstrations improve downstream imitation learning performance, leading to better generalizability. Notably, Bunny-VisionPro enables imitation learning with challenging multi-stage, long-horizon dexterous manipulation tasks, which have rarely been addressed in previous work. Our system's ability to handle bimanual manipulations while prioritizing safety and real-time performance makes it a powerful tool for advancing dexterous manipulation and imitation learning.