Enhancing Model-Based Step Adaptation for Push Recovery through Reinforcement Learning of Step Timing and Region

TL;DR

This paper presents a reinforcement learning-based method to dynamically adapt step timing and regions for humanoid robots, significantly improving push recovery capabilities under strong disturbances.

Contribution

It introduces a novel approach that expands footstep regions to non-convex areas and adapts timing in real time, enhancing robustness beyond traditional convex region constraints.

Findings

Expanded footstep regions enable recovery from larger pushes

Real-time timing adaptation improves disturbance rejection

Feasible footstep and trajectory planning enhances stability

Abstract

This paper introduces a new approach to enhance the robustness of humanoid walking under strong perturbations, such as substantial pushes. Effective recovery from external disturbances requires bipedal robots to dynamically adjust their stepping strategies, including footstep positions and timing. Unlike most advanced walking controllers that restrict footstep locations to a predefined convex region, substantially limiting recoverable disturbances, our method leverages reinforcement learning to dynamically adjust the permissible footstep region, expanding it to a larger, effectively non-convex area and allowing cross-over stepping, which is crucial for counteracting large lateral pushes. Additionally, our method adapts footstep timing in real time to further extend the range of recoverable disturbances. Based on these adjustments, feasible footstep positions and DCM trajectory are planned by solving a QP. Finally, we employ a DCM controller and an inverse dynamics whole-body control framework to ensure the robot effectively follows the trajectory.