CLONE: Closed-Loop Whole-Body Humanoid Teleoperation for Long-Horizon Tasks

TL;DR

CLONE is a novel closed-loop, MoE-based humanoid teleoperation system that achieves high-fidelity, long-duration, coordinated whole-body control with minimal positional drift using real-time feedback from head and hand tracking.

Contribution

This work introduces CLONE, the first system to combine closed-loop error correction with MoE learning for stable, precise, long-horizon humanoid teleoperation.

Findings

Maintains minimal positional drift over long trajectories.

Enables complex coordinated movements like object pickup.

Outperforms previous open-loop and stability-focused methods.

Abstract

Humanoid teleoperation plays a vital role in demonstrating and collecting data for complex humanoid-scene interactions. However, current teleoperation systems face critical limitations: they decouple upper- and lower-body control to maintain stability, restricting natural coordination, and operate open-loop without real-time position feedback, leading to accumulated drift. The fundamental challenge is achieving precise, coordinated whole-body teleoperation over extended durations while maintaining accurate global positioning. Here we show that an MoE-based teleoperation system, CLONE, with closed-loop error correction enables unprecedented whole-body teleoperation fidelity, maintaining minimal positional drift over long-range trajectories using only head and hand tracking from an MR headset. Unlike previous methods that either sacrifice coordination for stability or suffer from unbounded drift, CLONE learns diverse motion skills while preventing tracking error accumulation through real-time feedback, enabling complex coordinated movements such as ``picking up objects from the ground.'' These results establish a new milestone for whole-body humanoid teleoperation for long-horizon humanoid-scene interaction tasks.