A Gait Driven Reinforcement Learning Framework for Humanoid Robots

TL;DR

This paper introduces a real-time gait planning and reinforcement learning framework for humanoid robots, utilizing a novel hybrid pendulum model to improve gait efficiency and reduce learning time.

Contribution

A new gait planner using hybrid inverted pendulums combined with reinforcement learning for efficient, real-time humanoid robot gait generation.

Findings

Enhanced gait stability demonstrated in simulations.

Reduced learning time compared to traditional methods.

Improved locomotion performance in experiments.

Abstract

This paper presents a real-time gait driven training framework for humanoid robots. First, we introduce a novel gait planner that incorporates dynamics to design the desired joint trajectory. In the gait design process, the 3D robot model is decoupled into two 2D models, which are then approximated as hybrid inverted pendulums (H-LIP) for trajectory planning. The gait planner operates in parallel in real time within the robot's learning environment. Second, based on this gait planner, we design three effective reward functions within a reinforcement learning framework, forming a reward composition to achieve periodic bipedal gait. This reward composition reduces the robot's learning time and enhances locomotion performance. Finally, a gait design example, along with simulation and experimental comparisons, is presented to demonstrate the effectiveness of the proposed method.