MS-PPO: Morphological-Symmetry-Equivariant Policy for Legged Robot Locomotion

TL;DR

This paper introduces MS-PPO, a novel policy learning framework for legged robots that encodes morphological symmetry and kinematic structure, leading to improved training stability, generalization, and efficiency in diverse locomotion tasks.

Contribution

MS-PPO is the first to integrate morphological symmetry directly into a graph neural network policy for legged robots, eliminating the need for reward shaping or data augmentation.

Findings

MS-PPO outperforms state-of-the-art baselines in simulation and hardware.

It achieves higher sample efficiency and better symmetry generalization.

The approach enhances training stability across various locomotion tasks.

Abstract

Reinforcement learning has recently enabled impressive locomotion capabilities on legged robots; however, most policy architectures remain morphology- and symmetry-agnostic, leading to inefficient training and limited generalization. This work introduces MS-PPO, a morphological-symmetry-equivariant policy learning framework that encodes robot kinematic structure and morphological symmetries directly into the policy network. We construct a morphology-informed graph neural architecture that is provably equivariant with respect to the robot's morphological symmetry group actions, ensuring consistent policy responses under symmetric states while maintaining invariance in value estimation. This design eliminates the need for tedious reward shaping or costly data augmentation, which are typically required to enforce symmetry. We evaluate MS-PPO in simulation on Unitree Go2 and Xiaomi CyberDog2 robots across diverse locomotion tasks, including trotting, pronking, slope walking, and bipedal turning, and further deploy the learned policies on hardware. Extensive experiments show that MS-PPO achieves superior training stability, symmetry generalization ability, and sample efficiency in challenging locomotion tasks, compared to state-of-the-art baselines. These findings demonstrate that embedding both kinematic structure and morphological symmetry into policy learning provides a powerful inductive bias for legged robot locomotion control. Our code will be made publicly available at https://lunarlab-gatech.github.io/MS-PPO/.