Dynamics Modeling using Visual Terrain Features for High-Speed Autonomous Off-Road Driving

TL;DR

This paper introduces a visual terrain feature-based dynamics model for high-speed off-road autonomous driving, enabling accurate prediction of changing vehicle dynamics over unstructured terrain for improved planning.

Contribution

It presents a hybrid visual-based dynamics prediction model using a pre-trained foundation model and an end-to-end training architecture for real-time environment mapping.

Findings

Validated on extensive off-road driving dataset

Achieved accurate dynamics prediction over diverse terrains

Enabled improved autonomous navigation planning

Abstract

Rapid autonomous traversal of unstructured terrain is essential for scenarios such as disaster response, search and rescue, or planetary exploration. As a vehicle navigates at the limit of its capabilities over extreme terrain, its dynamics can change suddenly and dramatically. For example, high-speed and varying terrain can affect parameters such as traction, tire slip, and rolling resistance. To achieve effective planning in such environments, it is crucial to have a dynamics model that can accurately anticipate these conditions. In this work, we present a hybrid model that predicts the changing dynamics induced by the terrain as a function of visual inputs. We leverage a pre-trained visual foundation model (VFM) DINOv2, which provides rich features that encode fine-grained semantic information. To use this dynamics model for planning, we propose an end-to-end training architecture for a projection distance independent feature encoder that compresses the information from the VFM, enabling the creation of a lightweight map of the environment at runtime. We validate our architecture on an extensive dataset (hundreds of kilometers of aggressive off-road driving) collected across multiple locations as part of the DARPA Robotic Autonomy in Complex Environments with Resiliency (RACER) program. https://www.youtube.com/watch?v=dycTXxEosMk