ExBody2: Advanced Expressive Humanoid Whole-Body Control

TL;DR

This paper introduces ExBody2, a novel humanoid robot control method that enables expressive, stable, and robust whole-body motions by training on human and simulated data, then transferring to real robots.

Contribution

The paper presents a new decoupling technique for velocity and landmark tracking, along with a two-step training process for effective real-world deployment of expressive humanoid motions.

Findings

Improved tracking performance after fine-tuning.

Successful deployment of walking, crouching, and dancing motions.

Trade-off analysis between versatility and motion accuracy.

Abstract

This paper tackles the challenge of enabling real-world humanoid robots to perform expressive and dynamic whole-body motions while maintaining overall stability and robustness. We propose Advanced Expressive Whole-Body Control (Exbody2), a method for producing whole-body tracking controllers that are trained on both human motion capture and simulated data and then transferred to the real world. We introduce a technique for decoupling the velocity tracking of the entire body from tracking body landmarks. We use a teacher policy to produce intermediate data that better conforms to the robot's kinematics and to automatically filter away infeasible whole-body motions. This two-step approach enabled us to produce a student policy that can be deployed on the robot that can walk, crouch, and dance. We also provide insight into the trade-off between versatility and the tracking performance on specific motions. We observed significant improvement of tracking performance after fine-tuning on a small amount of data, at the expense of the others.