Quadruped Parkour Learning: Sparsely Gated Mixture of Experts with Visual Input

TL;DR

This paper explores the use of sparsely gated mixture-of-experts architectures for vision-based robotic parkour, demonstrating significant performance improvements and better efficiency compared to standard MLP control policies.

Contribution

It introduces the application of MoE architectures to robotic control for parkour, showing enhanced performance and computational efficiency over traditional MLP approaches.

Findings

MoE policies achieve double the success rate in obstacle traversal.

Matching MoE performance with MLP requires 14.3% more computation.

MoE architectures offer a better trade-off between performance and efficiency.

Abstract

Robotic parkour provides a compelling benchmark for advancing locomotion over highly challenging terrain, including large discontinuities such as elevated steps. Recent approaches have demonstrated impressive capabilities, including dynamic climbing and jumping, but typically rely on sequential multilayer perceptron (MLP) architectures with densely activated layers. In contrast, sparsely gated mixture-of-experts (MoE) architectures have emerged in the large language model domain as an effective paradigm for improving scalability and performance by activating only a subset of parameters at inference time. In this work, we investigate the application of sparsely gated MoE architectures to vision-based robotic parkour. We compare control policies based on standard MLPs and MoE architectures under a controlled setting where the number of active parameters at inference time is matched. Experimental results on a real Unitree Go2 quadruped robot demonstrate clear performance gains, with the MoE policy achieving double the number of successful trials in traversing large obstacles compared to a standard MLP baseline. We further show that achieving comparable performance with a standard MLP requires scaling its parameter count to match that of the total MoE model, resulting in a 14.3\% increase in computation time. These results highlight that sparsely gated MoE architectures provide a favorable trade-off between performance and computational efficiency, enabling improved scaling of control policies for vision-based robotic parkour. An anonymized link to the codebase is https://osf.io/v2kqj/files/github?view_only=7977dee10c0a44769184498eaba72e44.