Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robot

TL;DR

This paper introduces a constrained reinforcement learning approach for controlling unstable point-foot bipedal robots like Bolt, enhancing their stability and robustness through novel techniques and sim-to-real transfer methods.

Contribution

It presents a new constrained reinforcement learning methodology with Constraints-as-Terminations and domain randomization for stable bipedal locomotion.

Findings

Improved balance and velocity control in experiments

Enhanced robustness to slips and pushes

Effective sim-to-real transfer demonstrated

Abstract

Bipedal locomotion is a key challenge in robotics, particularly for robots like Bolt, which have a point-foot design. This study explores the control of such underactuated robots using constrained reinforcement learning, addressing their inherent instability, lack of arms, and limited foot actuation. We present a methodology that leverages Constraints-as-Terminations and domain randomization techniques to enable sim-to-real transfer. Through a series of qualitative and quantitative experiments, we evaluate our approach in terms of balance maintenance, velocity control, and responses to slip and push disturbances. Additionally, we analyze autonomy through metrics like the cost of transport and ground reaction force. Our method advances robust control strategies for point-foot bipedal robots, offering insights into broader locomotion.