YOPO-Rally: A Sim-to-Real Single-Stage Planner for Off-Road Terrain

TL;DR

This paper introduces YOPO-Rally, a novel sim-to-real off-road navigation framework that uses a unified neural network for planning, trained in simulation and directly deployed in real-world forest terrains.

Contribution

The work extends the YOPO framework to off-road environments with a new simulator, terrain analysis, and a neural planner that transfers zero-shot from simulation to real-world off-road navigation.

Findings

High success rate in real-world forest navigation

Effective zero-shot transfer from simulation to real environments

Competitive performance with existing off-road navigation methods

Abstract

Off-road navigation remains challenging for autonomous robots due to the harsh terrain and clustered obstacles. In this letter, we extend the YOPO (You Only Plan Once) end-to-end navigation framework to off-road environments, explicitly focusing on forest terrains, consisting of a high-performance, multi-sensor supported off-road simulator YOPO-Sim, a zero-shot transfer sim-to-real planner YOPO-Rally, and an MPC controller. Built on the Unity engine, the simulator can generate randomized forest environments and export depth images and point cloud maps for expert demonstrations, providing competitive performance with mainstream simulators. Terrain Traversability Analysis (TTA) processes cost maps, generating expert trajectories represented as non-uniform cubic Hermite curves. The planner integrates TTA and the pathfinding into a single neural network that inputs the depth image, current velocity, and the goal vector, and outputs multiple trajectory candidates with costs. The planner is trained by behavior cloning in the simulator and deployed directly into the real-world without fine-tuning. Finally, a series of simulated and real-world experiments is conducted to validate the performance of the proposed framework.