MOVE: Multi-skill Omnidirectional Legged Locomotion with Limited View in 3D Environments

TL;DR

MOVE is an end-to-end learning framework enabling quadruped robots with limited view to perform omnidirectional locomotion and complex tasks in 3D environments, mimicking animal-like adaptability.

Contribution

The paper introduces a novel one-stage neural network that combines supervised and contrastive learning for multi-skill locomotion with limited view in 3D terrains.

Findings

Effective in simulation and real-world tests

Enables extreme climbing and leaping

Robust to visual obstructions

Abstract

Legged robots possess inherent advantages in traversing complex 3D terrains. However, previous work on low-cost quadruped robots with egocentric vision systems has been limited by a narrow front-facing view and exteroceptive noise, restricting omnidirectional mobility in such environments. While building a voxel map through a hierarchical structure can refine exteroception processing, it introduces significant computational overhead, noise, and delays. In this paper, we present MOVE, a one-stage end-to-end learning framework capable of multi-skill omnidirectional legged locomotion with limited view in 3D environments, just like what a real animal can do. When movement aligns with the robot's line of sight, exteroceptive perception enhances locomotion, enabling extreme climbing and leaping. When vision is obstructed or the direction of movement lies outside the robot's field of view, the robot relies on proprioception for tasks like crawling and climbing stairs. We integrate all these skills into a single neural network by introducing a pseudo-siamese network structure combining supervised and contrastive learning which helps the robot infer its surroundings beyond its field of view. Experiments in both simulations and real-world scenarios demonstrate the robustness of our method, broadening the operational environments for robotics with egocentric vision.