From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots

TL;DR

This paper introduces BumbleBee, a framework that combines motion clustering and sim-to-real adaptation to develop a unified, robust, and agile whole-body control system for humanoid robots capable of handling diverse behaviors.

Contribution

The paper presents BumbleBee, a novel expert-generalist learning framework that effectively integrates motion clustering and real-world adaptation for humanoid robot control.

Findings

BumbleBee achieves state-of-the-art performance in general whole-body control.

The framework demonstrates robustness and agility across multiple motion types.

Experiments validate effectiveness on both simulation and real robots.

Abstract

Achieving general agile whole-body control on humanoid robots remains a major challenge due to diverse motion demands and data conflicts. While existing frameworks excel in training single motion-specific policies, they struggle to generalize across highly varied behaviors due to conflicting control requirements and mismatched data distributions. In this work, we propose BumbleBee (BB), an expert-generalist learning framework that combines motion clustering and sim-to-real adaptation to overcome these challenges. BB first leverages an autoencoder-based clustering method to group behaviorally similar motions using motion features and motion descriptions. Expert policies are then trained within each cluster and refined with real-world data through iterative delta action modeling to bridge the sim-to-real gap. Finally, these experts are distilled into a unified generalist controller that preserves agility and robustness across all motion types. Experiments on two simulations and a real humanoid robot demonstrate that BB achieves state-of-the-art general whole-body control, setting a new benchmark for agile, robust, and generalizable humanoid performance in the real world. The project webpage is available at https://beingbeyond.github.io/BumbleBee/.