Commanding Humanoid by Free-form Language: A Large Language Action Model with Unified Motion Vocabulary

TL;DR

Humanoid-LLA is a large language action model that translates natural language into stable, diverse, and physically plausible humanoid robot motions, advancing human-robot interaction.

Contribution

It introduces a unified motion vocabulary and a two-stage fine-tuning framework combining supervised learning and reinforcement learning with physical feedback.

Findings

Achieves superior generalization to new language commands.

Generates diverse and physically stable motions.

Performs well in both simulation and real-world tests.

Abstract

Enabling humanoid robots to follow free-form natural language commands is a critical step toward seamless human-robot interaction and general-purpose embodied AI. However, existing methods remain limited, often constrained to simple instructions or forced to sacrifice motion diversity for physical plausibility. To address this gap, we present Humanoid-LLA, a Large Language Action model that translates unconstrained natural language directly into executable whole-body motions for humanoid robots. Our approach tackles two core challenges: paired language-humanoid motion data scarcity and physical instability. First, we bridge high-level language semantics with physically-grounded control by learning a unified human-humanoid motion vocabulary. Second, we introduce a novel two-stage fine-tuning framework that begins with supervised motion Chain-of-Thought learning, followed by reinforcement learning refined with physical feedback to ensure robustness and stability. Extensive evaluation in simulation and real-world cross-embodiment experiments demonstrates that Humanoid-LLA achieves superior generalization to novel language commands and diverse motion generation while maintaining high physical fidelity.