Embedding Classical Balance Control Principles in Reinforcement Learning for Humanoid Recovery

TL;DR

This paper introduces a reinforcement learning approach for humanoid robot recovery that embeds classical balance metrics into the policy, enabling robust, zero-shot recovery behaviors across various disturbances and environments.

Contribution

It presents a novel RL framework that incorporates classical balance principles as privileged inputs, significantly improving recovery success and generalization without reliance on reference trajectories.

Findings

Achieves 93.4% recovery rate in diverse scenarios

Embedding balance metrics is crucial for successful learning

Demonstrates cross-environment transfer and hardware applicability

Abstract

Humanoid robots remain vulnerable to falls and unrecoverable failure states, limiting their practical utility in unstructured environments. While reinforcement learning has demonstrated stand-up behaviors, existing approaches treat recovery as a pure task-reward problem without an explicit representation of the balance state. We present a unified RL policy that addresses this limitation by embedding classical balance metrics: capture point, center-of-mass state, and centroidal momentum, as privileged critic inputs and shaping rewards directly around these quantities during training, while the actor relies solely on proprioception for zero-shot hardware transfer. Without reference trajectories or scripted contacts, a single policy spans the full recovery spectrum: ankle and hip strategies for small disturbances, corrective stepping under large pushes, and compliant falling with multi-contact stand-up using the hands, elbows, and knees. Trained on the Unitree H1-2 in Isaac Lab, the policy achieves a 93.4% recovery rate across randomized initial poses and unscripted fall configurations. An ablation study shows that removing the balance-informed structure causes stand-up learning to fail entirely, confirming that these metrics provide a meaningful learning signal rather than incidental structure. Sim-to-sim transfer to MuJoCo and preliminary hardware experiments further demonstrate cross-environment generalization. These results show that embedding interpretable balance structure into the learning framework substantially reduces time spent in failure states and broadens the envelope of autonomous recovery.