Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuning

TL;DR

This paper presents a new framework combining multi-expert distillation and reinforcement learning fine-tuning to develop a versatile, robust legged robot locomotion policy capable of navigating diverse unstructured terrains.

Contribution

It introduces a novel multi-stage training process that integrates terrain-specific policies into a unified, adaptable locomotion controller using distillation and RL fine-tuning.

Findings

Significant performance improvements over existing methods.

Successful deployment on real-world robots demonstrating agility.

Robust navigation across diverse unstructured terrains.

Abstract

Legged robots are well-suited for navigating terrains inaccessible to wheeled robots, making them ideal for applications in search and rescue or space exploration. However, current control methods often struggle to generalize across diverse, unstructured environments. This paper introduces a novel framework for agile locomotion of legged robots by combining multi-expert distillation with reinforcement learning (RL) fine-tuning to achieve robust generalization. Initially, terrain-specific expert policies are trained to develop specialized locomotion skills. These policies are then distilled into a unified foundation policy via the DAgger algorithm. The distilled policy is subsequently fine-tuned using RL on a broader terrain set, including real-world 3D scans. The framework allows further adaptation to new terrains through repeated fine-tuning. The proposed policy leverages depth images as exteroceptive inputs, enabling robust navigation across diverse, unstructured terrains. Experimental results demonstrate significant performance improvements over existing methods in synthesizing multi-terrain skills into a single controller. Deployment on the ANYmal D robot validates the policy's ability to navigate complex environments with agility and robustness, setting a new benchmark for legged robot locomotion.