A New Dataset and Framework for Robust Road Surface Classification via Camera-IMU Fusion

TL;DR

This paper introduces a multimodal camera-IMU framework and a diverse ROAD dataset to improve road surface classification robustness across varying environmental conditions, outperforming previous methods.

Contribution

It presents a novel fusion framework with attention and gating mechanisms, and a comprehensive dataset capturing diverse real-world, adverse, and synthetic conditions for RSC.

Findings

Achieved +1.4 percentage points improvement on PVS benchmark.

Achieved +11.6 percentage points improvement on ROAD dataset.

Demonstrated robustness under nighttime, rain, and surface transitions.

Abstract

Road surface classification (RSC) is a key enabler for environment-aware predictive maintenance systems. However, existing RSC techniques often fail to generalize beyond narrow operational conditions due to limited sensing modalities and datasets that lack environmental diversity. This work addresses these limitations by introducing a multimodal framework that fuses images and inertial measurements using a lightweight bidirectional cross-attention module followed by an adaptive gating layer that adjusts modality contributions under domain shifts. Given the limitations of current benchmarks, especially regarding lack of variability, we introduce ROAD, a new dataset composed of three complementary subsets: (i) real-world multimodal recordings with RGB-IMU streams synchronized using a gold-standard industry datalogger, captured across diverse lighting, weather, and surface conditions; (ii) a large vision-only subset designed to assess robustness under adverse illumination and heterogeneous capture setups; and (iii) a synthetic subset generated to study out-of-distribution generalization in scenarios difficult to obtain in practice. Experiments show that our method achieves a +1.4 pp improvement over the previous state-of-the-art on the PVS benchmark and an +11.6 pp improvement on our multimodal ROAD subset, with consistently higher F1-scores on minority classes. The framework also demonstrates stable performance across challenging visual conditions, including nighttime, heavy rain, and mixed-surface transitions. These findings indicate that combining affordable camera and IMU sensors with multimodal attention mechanisms provides a scalable, robust foundation for road surface understanding, particularly relevant for regions where environmental variability and cost constraints limit the adoption of high-end sensing suites.