End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy

TL;DR

This paper presents an end-to-end reinforcement learning approach for humanoid robot navigation that directly processes LiDAR data, ensuring safety and comfort through formal safety constraints and human-centric motion rewards, successfully transferring from simulation to real robots.

Contribution

It introduces a novel safety-aware reinforcement learning framework using Control Barrier Functions within a CMDP, enabling safe, comfortable, and robust humanoid robot navigation in complex environments.

Findings

Successful sim-to-real transfer on a humanoid robot

Robust navigation around static and dynamic obstacles

Enhanced safety and comfort in robot motions

Abstract

The deployment of humanoid robots in unstructured, human-centric environments requires navigation capabilities that extend beyond simple locomotion to include robust perception, provable safety, and socially aware behavior. Current reinforcement learning approaches are often limited by blind controllers that lack environmental awareness or by vision-based systems that fail to perceive complex 3D obstacles. In this work, we present an end-to-end locomotion policy that directly maps raw, spatio-temporal LiDAR point clouds to motor commands, enabling robust navigation in cluttered dynamic scenes. We formulate the control problem as a Constrained Markov Decision Process (CMDP) to formally separate safety from task objectives. Our key contribution is a novel methodology that translates the principles of Control Barrier Functions (CBFs) into costs within the CMDP, allowing a model-free Penalized Proximal Policy Optimization (P3O) to enforce safety constraints during training. Furthermore, we introduce a set of comfort-oriented rewards, grounded in human-robot interaction research, to promote motions that are smooth, predictable, and less intrusive. We demonstrate the efficacy of our framework through a successful sim-to-real transfer to a physical humanoid robot, which exhibits agile and safe navigation around both static and dynamic 3D obstacles.