FAME: Force-Adaptive RL for Expanding the Manipulation Envelope of a Full-Scale Humanoid

TL;DR

FAME introduces a force-adaptive reinforcement learning framework that enhances humanoid balance and manipulation capabilities by conditioning policies on learned force and pose encodings, enabling online adaptation without force sensors.

Contribution

The paper presents a novel RL approach that conditions on learned force and pose encodings, improving humanoid manipulation robustness under external forces without additional sensors.

Findings

FAME achieves 73.84% success rate in simulation, outperforming baselines.

The policy adapts online to external forces using estimated dynamics.

Deployment on a real humanoid demonstrates robustness in load scenarios.

Abstract

Maintaining balance under external hand forces is critical for humanoid bimanual manipulation, where interaction forces propagate through the kinematic chain and constrain the feasible manipulation envelope. We propose \textbf{FAME}, a force-adaptive reinforcement learning framework that conditions a standing policy on a learned latent context encoding upper-body joint configuration and bimanual interaction forces. During training, we apply diverse, spherically sampled 3D forces on each hand to inject disturbances in simulation together with an upper-body pose curriculum, exposing the policy to manipulation-induced perturbations across continuously varying arm configurations. At deployment, interaction forces are estimated from the robot dynamics and fed to the same encoder, enabling online adaptation without wrist force/torque sensors. In simulation across five fixed arm configurations with randomized hand forces and commanded base heights, FAME improves mean standing success to 73.84%, compared to 51.40% for the curriculum-only baseline and 29.44% for the base policy. We further deploy the learned policy on a full-scale Unitree H12 humanoid and evaluate robustness in representative load-interaction scenarios, including asymmetric single-arm load and symmetric bimanual load. Code and videos are available on https://fame10.github.io/Fame/