RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control

TL;DR

This paper introduces an innovative online bipedal footstep planning framework that combines MPC and reinforcement learning to enhance agility and robustness in humanoid robot locomotion, validated on the DRACO 3 robot.

Contribution

It presents a novel integration of MPC and RL for footstep planning, improving dynamic walking performance over traditional MPC methods.

Findings

Enhanced walking speed tracking

Reliable turning and terrain traversal

Improved robustness and stability

Abstract

This paper proposes an online bipedal footstep planning strategy that combines model predictive control (MPC) and reinforcement learning (RL) to achieve agile and robust bipedal maneuvers. While MPC-based foot placement controllers have demonstrated their effectiveness in achieving dynamic locomotion, their performance is often limited by the use of simplified models and assumptions. To address this challenge, we develop a novel foot placement controller that leverages a learned policy to bridge the gap between the use of a simplified model and the more complex full-order robot system. Specifically, our approach employs a unique combination of an ALIP-based MPC foot placement controller for sub-optimal footstep planning and the learned policy for refining footstep adjustments, enabling the resulting footstep policy to capture the robot's whole-body dynamics effectively. This integration synergizes the predictive capability of MPC with the flexibility and adaptability of RL. We validate the effectiveness of our framework through a series of experiments using the full-body humanoid robot DRACO 3. The results demonstrate significant improvements in dynamic locomotion performance, including better tracking of a wide range of walking speeds, enabling reliable turning and traversing challenging terrains while preserving the robustness and stability of the walking gaits compared to the baseline ALIP-based MPC approach.