Real-Time Whole-Body Teleoperation of a Humanoid Robot Using IMU-Based Motion Capture with Sim2Sim and Sim2Real Validation

TL;DR

This paper presents a real-time, low-latency whole-body teleoperation system for humanoid robots using IMU-based motion capture, validated through simulation and real-world experiments, enabling natural and synchronized robot movements.

Contribution

The authors develop a novel, offline-free control pipeline that maps human motion to a humanoid robot, bridging sim-to-real gaps with a scalable, practical approach.

Findings

System achieves stable, synchronized reproduction of diverse motions

Validated in simulation and on physical robot without modifications

Demonstrates real-time operation with low latency and broad motion repertoire

Abstract

Stable, low-latency whole-body teleoperation of humanoid robots is an open research challenge, complicated by kinematic mismatches between human and robot morphologies, accumulated inertial sensor noise, non-trivial control latency, and persistent sim-to-real transfer gaps. This paper presents a complete real-time whole-body teleoperation system that maps human motion, recorded with a Virdyn IMU-based full-body motion capture suit, directly onto a Unitree G1 humanoid robot. We introduce a custom motion-processing, kinematic retargeting, and control pipeline engineered for continuous, low-latency operation without any offline buffering or learning-based components. The system is first validated in simulation using the MuJoCo physics model of the Unitree G1 (sim2sim), and then deployed without modification on the physical platform (sim2real). Experimental results demonstrate stable, synchronized reproduction of a broad motion repertoire, including walking, standing, sitting, turning, bowing, and coordinated expressive full-body gestures. This work establishes a practical, scalable framework for whole-body humanoid teleoperation using commodity wearable motion capture hardware.