Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain

TL;DR

This paper introduces an end-to-end reinforcement learning system for wheeled robots to navigate complex, steep, and rugged terrains, using simulation and real-world experiments to demonstrate its effectiveness.

Contribution

It develops a novel RL approach with a terrain curriculum and reward function, enabling wheeled robots to learn navigation on challenging terrains without complex modeling.

Findings

RL achieves successful off-road navigation in simulation.

The approach transfers effectively to a physical robot platform.

RL outperforms traditional planning methods in rugged terrain.

Abstract

Off-road navigation on vertically challenging terrain, involving steep slopes and rugged boulders, presents significant challenges for wheeled robots both at the planning level to achieve smooth collision-free trajectories and at the control level to avoid rolling over or getting stuck. Considering the complex model of wheel-terrain interactions, we develop an end-to-end Reinforcement Learning (RL) system for an autonomous vehicle to learn wheeled mobility through simulated trial-and-error experiences. Using a custom-designed simulator built on the Chrono multi-physics engine, our approach leverages Proximal Policy Optimization (PPO) and a terrain difficulty curriculum to refine a policy based on a reward function to encourage progress towards the goal and penalize excessive roll and pitch angles, which circumvents the need of complex and expensive kinodynamic modeling, planning, and control. Additionally, we present experimental results in the simulator and deploy our approach on a physical Verti-4-Wheeler (V4W) platform, demonstrating that RL can equip conventional wheeled robots with previously unrealized potential of navigating vertically challenging terrain.