Reinforcement Learning Compensated Model Predictive Control for Off-road Driving on Unknown Deformable Terrain

TL;DR

This paper introduces a reinforcement learning-enhanced model predictive control framework for high-speed off-road driving on unknown deformable terrains, demonstrating superior performance and generalization over traditional methods in simulation.

Contribution

It presents a novel Actor-Critic reinforcement learning integrated with model predictive control specifically designed for complex, unknown deformable terrains in off-road autonomous driving.

Findings

Outperforms traditional controllers on various unknown terrains

Generalizes well to unseen terrain types with minimal training data

Converges faster and requires less data than purely learning-based controllers

Abstract

This study presents an Actor-Critic reinforcement learning Compensated Model Predictive Controller (AC2MPC) designed for high-speed, off-road autonomous driving on deformable terrains. Addressing the difficulty of modeling unknown tire-terrain interaction and ensuring real-time control feasibility and performance, this framework integrates deep reinforcement learning with a model predictive controller to manage unmodeled nonlinear dynamics. We evaluate the controller framework over constant and varying velocity profiles using high-fidelity simulator Project Chrono. Our findings demonstrate that our controller statistically outperforms standalone model-based and learning-based controllers over three unknown terrains that represent sandy deformable track, sandy and rocky track and cohesive clay-like deformable soil track. Despite varied and previously unseen terrain characteristics, this framework generalized well enough to track longitudinal reference speeds with the least error. Furthermore, this framework required significantly less training data compared to purely learning based controller, converging in fewer steps while delivering better performance. Even when under-trained, this controller outperformed the standalone controllers, highlighting its potential for safer and more efficient real-world deployment.