Watch Your Step: Learning Semantically-Guided Locomotion in Cluttered Environment

TL;DR

SemLoco is a reinforcement learning framework that enhances legged robot navigation in cluttered environments by integrating semantic understanding and pixel-wise safety inference for precise foot placement.

Contribution

The paper introduces SemLoco, a novel RL approach combining semantic maps and safety constraints to improve obstacle avoidance in dense, unstructured terrains.

Findings

SemLoco significantly reduces collisions in cluttered environments.

It enables safer navigation around sensitive objects.

Effective in complex real-world scenarios.

Abstract

Although legged robots demonstrate impressive mobility on rough terrain, using them safely in cluttered environments remains a challenge. A key issue is their inability to avoid stepping on low-lying objects, such as high-cost small devices or cables on flat ground. This limitation arises from a disconnection between high-level semantic understanding and low-level control, combined with errors in elevation maps during real-world operation. To address this, we introduce SemLoco, a Reinforcement Learning (RL) framework designed to avoid obstacles precisely in densely cluttered environments. SemLoco uses a two-stage RL approach that combines both soft and hard constraints. It performs pixel-wise foothold safety inference, which enables more accurate foot placement. Additionally, SemLoco integrates semantic map, allowing it to assign traversability costs instead of relying only on geometric data. SemLoco greatly reduces collisions and improves safety around sensitive objects, enabling reliable navigation in situations where traditional controllers would likely cause damage. Experimental results further show that SemLoco can be effectively applied to more complex, unstructured real-world environments. A demo video can be view at https://youtu.be/FSq-RSmIxOM.