Robust RL Control for Bipedal Locomotion with Closed Kinematic Chains

TL;DR

This paper introduces a reinforcement learning framework that explicitly models closed kinematic chain dynamics to improve the robustness and sim-to-real transfer of bipedal robot locomotion controllers, validated on a custom robot.

Contribution

The work presents a novel RL approach that incorporates closed-chain dynamics, symmetry-aware loss, and adversarial training for more robust bipedal locomotion control.

Findings

Enhanced policy robustness and stability across terrains

Significant improvement over simplified kinematic models

Successful validation on a custom-built bipedal robot

Abstract

Developing robust locomotion controllers for bipedal robots with closed kinematic chains presents unique challenges, particularly since most reinforcement learning (RL) approaches simplify these parallel mechanisms into serial models during training. We demonstrate that this simplification significantly impairs sim-to-real transfer by failing to capture essential aspects such as joint coupling, friction dynamics, and motor-space control characteristics. In this work, we present an RL framework that explicitly incorporates closed-chain dynamics and validate it on our custom-built robot TopA. Our approach enhances policy robustness through symmetry-aware loss functions, adversarial training, and targeted network regularization. Experimental results demonstrate that our integrated approach achieves stable locomotion across diverse terrains, significantly outperforming methods based on simplified kinematic models.