TTT-Parkour: Rapid Test-Time Training for Perceptive Robot Parkour

TL;DR

This paper introduces TTT-Parkour, a rapid test-time training framework that enables humanoid robots to master complex terrains by fine-tuning policies in less than 10 minutes using real-world reconstructions.

Contribution

It presents a novel real-to-sim-to-real approach with fast test-time training and an efficient geometry reconstruction pipeline for improved robot terrain traversal.

Findings

Robots successfully traverse complex obstacles like wedges and narrow beams.

Test-time training enhances zero-shot sim-to-real transfer performance.

The entire process takes less than 10 minutes on most terrains.

Abstract

Achieving highly dynamic humanoid parkour on unseen, complex terrains remains a challenge in robotics. Although general locomotion policies demonstrate capabilities across broad terrain distributions, they often struggle with arbitrary and highly challenging environments. To overcome this limitation, we propose a real-to-sim-to-real framework that leverages rapid test-time training (TTT) on novel terrains, significantly enhancing the robot's capability to traverse extremely difficult geometries. We adopt a two-stage end-to-end learning paradigm: a policy is first pre-trained on diverse procedurally generated terrains, followed by rapid fine-tuning on high-fidelity meshes reconstructed from real-world captures. Specifically, we develop a feed-forward, efficient, and high-fidelity geometry reconstruction pipeline using RGB-D inputs, ensuring both speed and quality during test-time training. We demonstrate that TTT-Parkour empowers humanoid robots to master complex obstacles, including wedges, stakes, boxes, trapezoids, and narrow beams. The whole pipeline of capturing, reconstructing, and test-time training requires less than 10 minutes on most tested terrains. Extensive experiments show that the policy after test-time training exhibits robust zero-shot sim-to-real transfer capability.