Walk the PLANC: Physics-Guided RL for Agile Humanoid Locomotion on Constrained Footholds

TL;DR

This paper presents a physics-guided reinforcement learning framework for humanoid robots to navigate constrained terrains with high accuracy and agility, combining structured planning with data-driven adaptation.

Contribution

It introduces a novel framework integrating a reduced-order stepping planner with RL using CLF rewards, enhancing reliability and precision in humanoid locomotion on complex terrains.

Findings

Achieved hardware-validated stepping-stone locomotion

Significantly improved reliability over model-free RL baselines

Produced accurate and agile movements on constrained terrains

Abstract

Bipedal humanoid robots must precisely coordinate balance, timing, and contact decisions when locomoting on constrained footholds such as stepping stones, beams, and planks -- even minor errors can lead to catastrophic failure. Classical optimization and control pipelines handle these constraints well but depend on highly accurate mathematical representations of terrain geometry, making them prone to error when perception is noisy or incomplete. Meanwhile, reinforcement learning has shown strong resilience to disturbances and modeling errors, yet end-to-end policies rarely discover the precise foothold placement and step sequencing required for discontinuous terrain. These contrasting limitations motivate approaches that guide learning with physics-based structure rather than relying purely on reward shaping. In this work, we introduce a locomotion framework in which a reduced-order stepping planner supplies dynamically consistent motion targets that steer the RL training process via Control Lyapunov Function (CLF) rewards. This combination of structured footstep planning and data-driven adaptation produces accurate, agile, and hardware-validated stepping-stone locomotion on a humanoid robot, substantially improving reliability compared to conventional model-free reinforcement-learning baselines.