TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

TL;DR

TeleMoMa is a modular teleoperation system that simplifies data collection for mobile manipulation robots by integrating multiple interfaces, enabling easier demonstration collection for imitation learning in real and simulated environments.

Contribution

The paper introduces TeleMoMa, a versatile, modular teleoperation system that unifies various human interfaces and lowers the barrier for collecting mobile manipulation demonstrations.

Findings

TeleMoMa successfully teleoperates multiple mobile robots in simulation and real-world.

Demonstrations collected with TeleMoMa improve imitation learning policies.

User studies show TeleMoMa is accessible for novice users.

Abstract

A critical bottleneck limiting imitation learning in robotics is the lack of data. This problem is more severe in mobile manipulation, where collecting demonstrations is harder than in stationary manipulation due to the lack of available and easy-to-use teleoperation interfaces. In this work, we demonstrate TeleMoMa, a general and modular interface for whole-body teleoperation of mobile manipulators. TeleMoMa unifies multiple human interfaces including RGB and depth cameras, virtual reality controllers, keyboard, joysticks, etc., and any combination thereof. In its more accessible version, TeleMoMa works using simply vision (e.g., an RGB-D camera), lowering the entry bar for humans to provide mobile manipulation demonstrations. We demonstrate the versatility of TeleMoMa by teleoperating several existing mobile manipulators - PAL Tiago++, Toyota HSR, and Fetch - in simulation and the real world. We demonstrate the quality of the demonstrations collected with TeleMoMa by training imitation learning policies for mobile manipulation tasks involving synchronized whole-body motion. Finally, we also show that TeleMoMa's teleoperation channel enables teleoperation on site, looking at the robot, or remote, sending commands and observations through a computer network, and perform user studies to evaluate how easy it is for novice users to learn to collect demonstrations with different combinations of human interfaces enabled by our system. We hope TeleMoMa becomes a helpful tool for the community enabling researchers to collect whole-body mobile manipulation demonstrations. For more information and video results, https://robin-lab.cs.utexas.edu/telemoma-web.