OTTR: Off-Road Trajectory Tracking using Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning algorithm for off-road trajectory tracking that effectively adapts to complex terrains using limited real-world data, outperforming traditional methods.

Contribution

It proposes a supervised-learning based adaptation method to bridge the sim-to-real gap in off-road RL without complex vehicle-environment modeling.

Findings

Achieves 30-50% reduction in cross track error

Uses only 30 minutes of real-world data

Outperforms standard ILQR approach

Abstract

In this work, we present a novel Reinforcement Learning (RL) algorithm for the off-road trajectory tracking problem. Off-road environments involve varying terrain types and elevations, and it is difficult to model the interaction dynamics of specific off-road vehicles with such a diverse and complex environment. Standard RL policies trained on a simulator will fail to operate in such challenging real-world settings. Instead of using a naive domain randomization approach, we propose an innovative supervised-learning based approach for overcoming the sim-to-real gap problem. Our approach efficiently exploits the limited real-world data available to adapt the baseline RL policy obtained using a simple kinematics simulator. This avoids the need for modeling the diverse and complex interaction of the vehicle with off-road environments. We evaluate the performance of the proposed algorithm using two different off-road vehicles, Warthog and Moose. Compared to the standard ILQR approach, our proposed approach achieves a 30% and 50% reduction in cross track error in Warthog and Moose, respectively, by utilizing only 30 minutes of real-world driving data.