HoRD: Robust Humanoid Control via History-Conditioned Reinforcement Learning and Online Distillation

TL;DR

HoRD introduces a two-stage reinforcement learning framework that enhances humanoid robot control robustness by combining history-conditioned adaptation with online distillation, enabling zero-shot transfer to unseen domains.

Contribution

The paper presents a novel two-stage framework that improves humanoid control robustness under domain shifts by integrating history-conditioned RL with online distillation into a transformer-based policy.

Findings

HoRD outperforms baselines in robustness and transfer.

The approach enables zero-shot adaptation to unseen domains.

It maintains high performance under external perturbations.

Abstract

Humanoid robots can suffer significant performance drops under small changes in dynamics, task specifications, or environment setup. We propose HoRD, a two-stage learning framework for robust humanoid control under domain shift. First, we train a high-performance teacher policy via history-conditioned reinforcement learning, where the policy infers latent dynamics context from recent state--action trajectories to adapt online to diverse randomized dynamics. Second, we perform online distillation to transfer the teacher's robust control capabilities into a transformer-based student policy that operates on sparse root-relative 3D joint keypoint trajectories. By combining history-conditioned adaptation with online distillation, HoRD enables a single policy to adapt zero-shot to unseen domains without per-domain retraining. Extensive experiments show HoRD outperforms strong baselines in robustness and transfer, especially under unseen domains and external perturbations. Code and project page are available at https://tonywang-0517.github.io/hord/.