Sim-to-Real of Humanoid Locomotion Policies via Joint Torque Space Perturbation Injection

TL;DR

This paper introduces a new sim-to-real transfer method for humanoid locomotion that injects neural network-generated, state-dependent torque perturbations during simulation, improving robustness without extra training.

Contribution

It presents a novel perturbation injection approach using neural networks to better simulate complex reality gaps in humanoid locomotion policies.

Findings

Enhanced robustness of policies in real-world tests

Better handling of complex, unseen reality gaps

No additional training required for the perturbation method

Abstract

This paper proposes a novel alternative to existing sim-to-real methods for training control policies with simulated experiences. Unlike prior methods that typically rely on domain randomization over a fixed finite set of parameters, the proposed approach injects state-dependent perturbations into the input joint torque during forward simulation. These perturbations are designed to simulate a broader spectrum of reality gaps than standard parameter randomization without requiring additional training. By using neural networks as flexible perturbation generators, the proposed method can represent complex, state-dependent uncertainties, such as nonlinear actuator dynamics and contact compliance, that parametric randomization cannot capture. Experimental results demonstrate that the proposed approach enables humanoid locomotion policies to achieve superior robustness against complex, unseen reality gaps in both simulation and real-world deployment.