STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

TL;DR

The STONE dataset offers a large-scale, multi-modal, annotated dataset for off-road robot navigation, enabling improved 3D traversability prediction through automated labeling and comprehensive sensor data.

Contribution

We introduce STONE, a scalable multi-modal dataset with an automated annotation pipeline for 3D traversability, covering diverse environments and conditions for off-road navigation research.

Findings

Provides dense terrain surface reconstructions from LiDAR data.

Establishes a benchmark for voxel-level 3D traversability prediction.

Demonstrates effective multi-modal perception for off-road navigation.

Abstract

Reliable off-road navigation requires accurate estimation of traversable regions and robust perception under diverse terrain and sensing conditions. However, existing datasets lack both scalability and multi-modality, which limits progress in 3D traversability prediction. In this work, we introduce STONE, a large-scale multi-modal dataset for off-road navigation. STONE provides (1) trajectory-guided 3D traversability maps generated by a fully automated, annotation-free pipeline, and (2) comprehensive surround-view sensing with synchronized 128-channel LiDAR, six RGB cameras, and three 4D imaging radars. The dataset covers a wide range of environments and conditions, including day and night, grasslands, farmlands, construction sites, and lakes. Our auto-labeling pipeline reconstructs dense terrain surfaces from LiDAR scans, extracts geometric attributes such as slope, elevation, and roughness, and assigns traversability labels beyond the robot's trajectory using a Mahalanobis-distance-based criterion. This design enables scalable, geometry-aware ground-truth construction without manual annotation. Finally, we establish a benchmark for voxel-level 3D traversability prediction and provide strong baselines under both single-modal and multi-modal settings. STONE is available at: https://konyul.github.io/STONE-dataset/