Keep on Going: Learning Robust Humanoid Motion Skills via Selective Adversarial Training

TL;DR

This paper introduces SA2RT, a novel adversarial training method that enhances the robustness of humanoid robot motion policies by selectively attacking vulnerable states, leading to significant improvements in terrain traversal and tracking accuracy.

Contribution

The paper presents a new selective adversarial attack framework for robust reinforcement learning of humanoid motion skills, avoiding overfitting and improving long-term stability.

Findings

40% increase in terrain traversal success rate

32% reduction in trajectory tracking error

Enhanced long horizon mobility and robustness

Abstract

Humanoid robots are expected to operate reliably over long horizons while executing versatile whole-body skills. Yet Reinforcement Learning (RL) motion policies typically lose stability under prolonged operation, sensor/actuator noise, and real world disturbances. In this work, we propose a Selective Adversarial Attack for Robust Training (SA2RT) to enhance the robustness of motion skills. The adversary is learned to identify and sparsely perturb the most vulnerable states and actions under an attack-budget constraint, thereby exposing true weakness without inducing conservative overfitting. The resulting non-zero sum, alternating optimization continually strengthens the motion policy against the strongest discovered attacks. We validate our approach on the Unitree G1 humanoid robot across perceptive locomotion and whole-body control tasks. Experimental results show that adversarially trained policies improve the terrain traversal success rate by 40%, reduce the trajectory tracking error by 32%, and maintain long horizon mobility and tracking performance. Together, these results demonstrate that selective adversarial attacks are an effective driver for learning robust, long horizon humanoid motion skills.