PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representations

TL;DR

This paper introduces PvP, a contrastive learning framework that enhances data efficiency in humanoid robot control by leveraging proprioceptive and privileged states, supported by a new evaluation platform.

Contribution

The paper presents PvP, a novel contrastive learning approach that improves sample efficiency in humanoid robot learning without hand-crafted data augmentations.

Findings

PvP significantly outperforms baseline SRL methods in experiments.

PvP accelerates policy learning and enhances final task performance.

The SRL4Humanoid framework enables systematic evaluation of SRL methods.

Abstract

Achieving efficient and robust whole-body control (WBC) is essential for enabling humanoid robots to perform complex tasks in dynamic environments. Despite the success of reinforcement learning (RL) in this domain, its sample inefficiency remains a significant challenge due to the intricate dynamics and partial observability of humanoid robots. To address this limitation, we propose PvP, a Proprioceptive-Privileged contrastive learning framework that leverages the intrinsic complementarity between proprioceptive and privileged states. PvP learns compact and task-relevant latent representations without requiring hand-crafted data augmentations, enabling faster and more stable policy learning. To support systematic evaluation, we develop SRL4Humanoid, the first unified and modular framework that provides high-quality implementations of representative state representation learning (SRL) methods for humanoid robot learning. Extensive experiments on the LimX Oli robot across velocity tracking and motion imitation tasks demonstrate that PvP significantly improves sample efficiency and final performance compared to baseline SRL methods. Our study further provides practical insights into integrating SRL with RL for humanoid WBC, offering valuable guidance for data-efficient humanoid robot learning.