Rhythm: Learning Interactive Whole-Body Control for Dual Humanoids

TL;DR

Rhythm is a unified framework enabling dual-humanoid systems to perform complex, physically plausible interactions like hugging and dancing, bridging simulation and real-world deployment.

Contribution

The paper introduces Rhythm, a novel integrated system combining motion retargeting, reinforcement learning, and deployment techniques for interactive dual-humanoid control.

Findings

Successfully transferred diverse behaviors from simulation to physical robots.

Achieved robust, interactive whole-body control in real-world experiments.

Enabled complex interactions like hugging and dancing with dual humanoids.

Abstract

Realizing interactive whole-body control for multi-humanoid systems is critical for unlocking complex collaborative capabilities in shared environments. Although recent advancements have significantly enhanced the agility of individual robots, bridging the gap to physically coupled multi-humanoid interaction remains challenging, primarily due to severe kinematic mismatches and complex contact dynamics. To address this, we introduce Rhythm, the first unified framework enabling real-world deployment of dual-humanoid systems for complex, physically plausible interactions. Our framework integrates three core components: (1) an Interaction-Aware Motion Retargeting (IAMR) module that generates feasible humanoid interaction references from human data; (2) an Interaction-Guided Reinforcement Learning (IGRL) policy that masters coupled dynamics via graph-based rewards; and (3) a real-world deployment system that enables robust transfer of dual-humanoid interaction. Extensive experiments on physical Unitree G1 robots demonstrate that our framework achieves robust interactive whole-body control, successfully transferring diverse behaviors such as hugging and dancing from simulation to reality.