Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation

TL;DR

This paper presents a two-stage training pipeline for robotic athletic loco-manipulation that effectively bridges the sim-to-real gap, enabling robots to perform dynamic, goal-oriented tasks with high fidelity from simulation to reality.

Contribution

It introduces the Unsupervised Actuator Net (UAN) for sim-to-real transfer without torque sensing and a pre-training strategy using reference trajectories to improve exploration.

Findings

Robots can lift, throw, and drag with high accuracy from simulation to real-world.

UAN reduces reward hacking and enhances transfer robustness.

Pre-training with reference trajectories accelerates learning and improves task performance.

Abstract

Achieving athletic loco-manipulation on robots requires moving beyond traditional tracking rewards - which simply guide the robot along a reference trajectory - to task rewards that drive truly dynamic, goal-oriented behaviors. Commands such as "throw the ball as far as you can" or "lift the weight as quickly as possible" compel the robot to exhibit the agility and power inherent in athletic performance. However, training solely with task rewards introduces two major challenges: these rewards are prone to exploitation (reward hacking), and the exploration process can lack sufficient direction. To address these issues, we propose a two-stage training pipeline. First, we introduce the Unsupervised Actuator Net (UAN), which leverages real-world data to bridge the sim-to-real gap for complex actuation mechanisms without requiring access to torque sensing. UAN mitigates reward hacking by ensuring that the learned behaviors remain robust and transferable. Second, we use a pre-training and fine-tuning strategy that leverages reference trajectories as initial hints to guide exploration. With these innovations, our robot athlete learns to lift, throw, and drag with remarkable fidelity from simulation to reality.