TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation

TL;DR

TeleOpBench is a comprehensive, simulator-based benchmark for evaluating and comparing various dual-arm dexterous teleoperation systems across multiple tasks, validated by real-world experiments.

Contribution

Introduces TeleOpBench, a unified, high-fidelity simulation benchmark for bimanual teleoperation, enabling fair comparison and validation of diverse hardware and control modalities.

Findings

Strong correlation between simulation and real-world performance.

Benchmark covers a broad spectrum of tasks and modalities.

Provides an extensible platform for future research.

Abstract

Teleoperation is a cornerstone of embodied-robot learning, and bimanual dexterous teleoperation in particular provides rich demonstrations that are difficult to obtain with fully autonomous systems. While recent studies have proposed diverse hardware pipelines-ranging from inertial motion-capture gloves to exoskeletons and vision-based interfaces-there is still no unified benchmark that enables fair, reproducible comparison of these systems. In this paper, we introduce TeleOpBench, a simulator-centric benchmark tailored to bimanual dexterous teleoperation. TeleOpBench contains 30 high-fidelity task environments that span pick-and-place, tool use, and collaborative manipulation, covering a broad spectrum of kinematic and force-interaction difficulty. Within this benchmark we implement four representative teleoperation modalities-(i) MoCap, (ii) VR device, (iii) arm-hand exoskeletons, and (iv) monocular vision tracking-and evaluate them with a common protocol and metric suite. To validate that performance in simulation is predictive of real-world behavior, we conduct mirrored experiments on a physical dual-arm platform equipped with two 6-DoF dexterous hands. Across 10 held-out tasks we observe a strong correlation between simulator and hardware performance, confirming the external validity of TeleOpBench. TeleOpBench establishes a common yardstick for teleoperation research and provides an extensible platform for future algorithmic and hardware innovation. Codes is now available at https://github.com/cyjdlhy/TeleOpBench .