TartanDrive: A Large-Scale Dataset for Learning Off-Road Dynamics Models

TL;DR

TartanDrive is the largest real-world multi-modal off-road driving dataset, enabling improved learning of dynamics models for challenging terrains and benchmarking current methods.

Contribution

The paper introduces TartanDrive, a comprehensive large-scale dataset with diverse modalities for off-road driving, and evaluates state-of-the-art models on it.

Findings

Multi-modal models outperform single-modality in off-road dynamics prediction.

Current neural architectures have limitations for off-road driving tasks.

Dataset facilitates future research in off-road autonomous driving.

Abstract

We present TartanDrive, a large scale dataset for learning dynamics models for off-road driving. We collected a dataset of roughly 200,000 off-road driving interactions on a modified Yamaha Viking ATV with seven unique sensing modalities in diverse terrains. To the authors' knowledge, this is the largest real-world multi-modal off-road driving dataset, both in terms of number of interactions and sensing modalities. We also benchmark several state-of-the-art methods for model-based reinforcement learning from high-dimensional observations on this dataset. We find that extending these models to multi-modality leads to significant performance on off-road dynamics prediction, especially in more challenging terrains. We also identify some shortcomings with current neural network architectures for the off-road driving task. Our dataset is available at https://github.com/castacks/tartan_drive.