OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation

TL;DR

OPEN TEACH is an accessible, VR-based teleoperation system that enables intuitive control of various robots for manipulation tasks, fostering data collection and policy learning in robotics research.

Contribution

It introduces a versatile, open-source teleoperation platform using affordable VR hardware that supports multiple robot types and improves user control experience.

Findings

Demonstrated control over 38 diverse tasks across different robots.

User study shows significant performance improvements over existing frameworks.

Collected data effectively used for policy learning in dexterous manipulation.

Abstract

Open-sourced, user-friendly tools form the bedrock of scientific advancement across disciplines. The widespread adoption of data-driven learning has led to remarkable progress in multi-fingered dexterity, bimanual manipulation, and applications ranging from logistics to home robotics. However, existing data collection platforms are often proprietary, costly, or tailored to specific robotic morphologies. We present OPEN TEACH, a new teleoperation system leveraging VR headsets to immerse users in mixed reality for intuitive robot control. Built on the affordable Meta Quest 3, which costs $500, OPEN TEACH enables real-time control of various robots, including multi-fingered hands and bimanual arms, through an easy-to-use app. Using natural hand gestures and movements, users can manipulate robots at up to 90Hz with smooth visual feedback and interface widgets offering closeup environment views. We demonstrate the versatility of OPEN TEACH across 38 tasks on different robots. A comprehensive user study indicates significant improvement in teleoperation capability over the AnyTeleop framework. Further experiments exhibit that the collected data is compatible with policy learning on 10 dexterous and contact-rich manipulation tasks. Currently supporting Franka, xArm, Jaco, and Allegro platforms, OPEN TEACH is fully open-sourced to promote broader adoption. Videos are available at https://open-teach.github.io/.