VertiSelector: Automatic Curriculum Learning for Wheeled Mobility on Vertically Challenging Terrain

TL;DR

VertiSelector is an automatic curriculum learning framework that improves reinforcement learning for wheeled robots on challenging terrains by focusing training on difficult areas, leading to better efficiency and real-world generalization.

Contribution

We introduce VertiSelector, a novel automatic curriculum learning method that dynamically prioritizes challenging terrains to enhance RL training efficiency and transferability.

Findings

Achieves 23.08% success rate improvement in simulation.

Enhances sample efficiency during training.

Demonstrates robust real-world generalization.

Abstract

Reinforcement Learning (RL) has the potential to enable extreme off-road mobility by circumventing complex kinodynamic modeling, planning, and control by simulated end-to-end trial-and-error learning experiences. However, most RL methods are sample-inefficient when training in a large amount of manually designed simulation environments and struggle at generalizing to the real world. To address these issues, we introduce VertiSelector (VS), an automatic curriculum learning framework designed to enhance learning efficiency and generalization by selectively sampling training terrain. VS prioritizes vertically challenging terrain with higher Temporal Difference (TD) errors when revisited, thereby allowing robots to learn at the edge of their evolving capabilities. By dynamically adjusting the sampling focus, VS significantly boosts sample efficiency and generalization within the VW-Chrono simulator built on the Chrono multi-physics engine. Furthermore, we provide simulation and physical results using VS on a Verti-4-Wheeler platform. These results demonstrate that VS can achieve 23.08% improvement in terms of success rate by efficiently sampling during training and robustly generalizing to the real world.