No More Marching: Learning Humanoid Locomotion for Short-Range SE(2) Targets

TL;DR

This paper presents a reinforcement learning method for humanoid robots to efficiently reach short-range SE(2) targets, emphasizing natural movement and energy efficiency, with a new reward function and benchmarking framework.

Contribution

It introduces a constellation-based reward function for direct pose optimization and a benchmarking framework for evaluating short-range humanoid locomotion.

Findings

The approach outperforms standard methods in energy and time metrics.

Successful transfer of learned locomotion from simulation to hardware.

The reward design significantly improves target-oriented movement efficiency.

Abstract

Humanoids operating in real-world workspaces must frequently execute task-driven, short-range movements to SE(2) target poses. To be practical, these transitions must be fast, robust, and energy efficient. While learning-based locomotion has made significant progress, most existing methods optimize for velocity-tracking rather than direct pose reaching, resulting in inefficient, marching-style behavior when applied to short-range tasks. In this work, we develop a reinforcement learning approach that directly optimizes humanoid locomotion for SE(2) targets. Central to this approach is a new constellation-based reward function that encourages natural and efficient target-oriented movement. To evaluate performance, we introduce a benchmarking framework that measures energy consumption, time-to-target, and footstep count on a distribution of SE(2) goals. Our results show that the proposed approach consistently outperforms standard methods and enables successful transfer from simulation to hardware, highlighting the importance of targeted reward design for practical short-range humanoid locomotion.