Mechanical Intelligence-Aware Curriculum Reinforcement Learning for Humanoids with Parallel Actuation

TL;DR

This paper introduces a novel RL framework that explicitly models parallel actuation mechanisms in humanoid robots, leading to more accurate control policies and improved real-world performance over traditional serial approximations.

Contribution

It develops simulation methods for parallel mechanisms and integrates them into a curriculum RL framework for humanoids, enhancing policy accuracy and transferability.

Findings

Better surface generalization in real-world tests

Improved zero-shot deployment performance

Native simulation of closed-chain constraints

Abstract

Reinforcement learning (RL) has enabled advances in humanoid robot locomotion, yet most learning frameworks do not account for mechanical intelligence embedded in parallel actuation mechanisms due to limitations in simulator support for closed kinematic chains. This omission can lead to inaccurate motion modeling and suboptimal policies, particularly for robots with high actuation complexity. This paper presents general formulations and simulation methods for three types of parallel mechanisms: a differential pulley, a five-bar linkage, and a four-bar linkage, and trains a parallel-mechanism aware policy through an end-to-end curriculum RL framework for BRUCE, a kid-sized humanoid robot. Unlike prior approaches that rely on simplified serial approximations, we simulate all closed-chain constraints natively using GPU-accelerated MuJoCo (MJX), preserving the hardware's mechanical nonlinear properties during training. We benchmark our RL approach against a model predictive controller (MPC), demonstrating better surface generalization and performance in real-world zero-shot deployment. This work highlights the computational approaches and performance benefits of fully simulating parallel mechanisms in end-to-end learning pipelines for legged humanoids. Project codes with parallel mechanisms: https://github.com/alvister88/og_bruce