HALO:Closing Sim-to-Real Gap for Heavy-loaded Humanoid Agile Motion Skills via Differentiable Simulation

TL;DR

This paper introduces HALO, a two-stage differentiable simulation framework that effectively reduces the sim-to-real gap for humanoid robots carrying unknown payloads, enabling zero-shot transfer of learned skills.

Contribution

The paper presents a novel two-stage gradient-based system identification method using MuJoCo XLA to calibrate robot models and payload mass distribution, improving sim-to-real transfer for heavy-loaded humanoid motion.

Findings

Enhanced motion tracking accuracy in real-world tests

More precise parameter identification compared to baselines

Significantly improved agility and robustness of policies

Abstract

Humanoid robots deployed in real-world scenarios often need to carry unknown payloads, which introduce significant mismatch and degrade the effectiveness of simulation-to-reality reinforcement learning methods. To address this challenge, we propose a two-stage gradient-based system identification framework built on the differentiable simulator MuJoCo XLA. The first stage calibrates the nominal robot model using real-world data to reduce intrinsic sim-to-real discrepancies, while the second stage further identifies the mass distribution of the unknown payload. By explicitly reducing structured model bias prior to policy training, our approach enables zero-shot transfer of reinforcement learning policies to hardware under heavy-load conditions. Extensive simulation and real-world experiments demonstrate more precise parameter identification, improved motion tracking accuracy, and substantially enhanced agility and robustness compared to existing baselines. Project Page: https://mwondering.github.io/halo-humanoid/