Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments

TL;DR

This paper introduces Omni-Perception, an end-to-end LiDAR-based perception and control system for legged robots, enabling robust omnidirectional collision avoidance in complex, dynamic environments through a novel hierarchical risk assessment network.

Contribution

It presents a novel perception module, PD-RiskNet, and a high-fidelity LiDAR simulation toolkit for scalable training and effective sim-to-real transfer.

Findings

Robust navigation in complex environments with static and dynamic obstacles.

Superior collision avoidance compared to map-based approaches.

Effective sim-to-real transfer demonstrated in real-world experiments.

Abstract

Agile locomotion in complex 3D environments requires robust spatial awareness to safely avoid diverse obstacles such as aerial clutter, uneven terrain, and dynamic agents. Depth-based perception approaches often struggle with sensor noise, lighting variability, computational overhead from intermediate representations (e.g., elevation maps), and difficulties with non-planar obstacles, limiting performance in unstructured environments. In contrast, direct integration of LiDAR sensing into end-to-end learning for legged locomotion remains underexplored. We propose Omni-Perception, an end-to-end locomotion policy that achieves 3D spatial awareness and omnidirectional collision avoidance by directly processing raw LiDAR point clouds. At its core is PD-RiskNet (Proximal-Distal Risk-Aware Hierarchical Network), a novel perception module that interprets spatio-temporal LiDAR data for environmental risk assessment. To facilitate efficient policy learning, we develop a high-fidelity LiDAR simulation toolkit with realistic noise modeling and fast raycasting, compatible with platforms such as Isaac Gym, Genesis, and MuJoCo, enabling scalable training and effective sim-to-real transfer. Learning reactive control policies directly from raw LiDAR data enables the robot to navigate complex environments with static and dynamic obstacles more robustly than approaches relying on intermediate maps or limited sensing. We validate Omni-Perception through real-world experiments and extensive simulation, demonstrating strong omnidirectional avoidance capabilities and superior locomotion performance in highly dynamic environments.