RoAD Benchmark: How LiDAR Models Fail under Coupled Domain Shifts and Label Evolution

TL;DR

The paper introduces RoAD, a benchmark for evaluating LiDAR model robustness under combined domain shifts and label evolution in autonomous driving, highlighting key failure modes.

Contribution

It presents a new benchmark and analysis framework for assessing how LiDAR models handle evolving environments and labels in autonomous driving datasets.

Findings

Limited transferability in subclass refinement and unseen-class insertion scenarios.

Accelerated forgetting during continual adaptation due to feature collapse.

Identifies key failure modes in LiDAR-based object classification under dataset shifts.

Abstract

For 3D perception systems to operate reliably in real-world environments, they must remain robust to evolving sensor characteristics and changes in object taxonomies. However, existing adaptive learning paradigms struggle in LiDAR settings where domain shifts and label-space evolution occur simultaneously. We introduce \textbf{Robust Autonomous Driving under Dataset shifts (RoAD)}, a benchmark for evaluating model robustness in LiDAR-based object classification under intertwined domain shifts and label evolution, including subclass refinement, unseen-class insertion, and label expansion. RoAD evaluates three learning scenarios with increasing adaptation, from fixed representations (zero-shot transfer and linear probing) to sequential updates (continual learning). Experiments span large-scale autonomous driving datasets, including Waymo, nuScenes, and Argoverse2. Our analysis identifies central failure modes: (i) \textit{limited transferability} under subclass refinement and unseen-class insertion, and on non-vehicle class; and (ii) \textit{accelerated forgetting during continual adaptation}, driven by feature collapse and self-supervised learning objectives.