RoadMamba: A Dual Branch Visual State Space Model for Road Surface Classification

TL;DR

RoadMamba introduces a dual branch visual model that combines local texture and global semantics for improved road surface classification, achieving state-of-the-art results on a large-scale dataset.

Contribution

The paper presents a novel dual state space model with dual attention fusion and auxiliary loss for enhanced local and global feature extraction in road surface classification.

Findings

Achieves state-of-the-art accuracy on a large-scale dataset.

Effectively combines local texture and global semantics.

Outperforms existing models in road surface classification.

Abstract

Acquiring the road surface conditions in advance based on visual technologies provides effective information for the planning and control system of autonomous vehicles, thus improving the safety and driving comfort of the vehicles. Recently, the Mamba architecture based on state-space models has shown remarkable performance in visual processing tasks, benefiting from the efficient global receptive field. However, existing Mamba architectures struggle to achieve state-of-the-art visual road surface classification due to their lack of effective extraction of the local texture of the road surface. In this paper, we explore for the first time the potential of visual Mamba architectures for road surface classification task and propose a method that effectively combines local and global perception, called RoadMamba. Specifically, we utilize the Dual State Space Model (DualSSM) to effectively extract the global semantics and local texture of the road surface and decode and fuse the dual features through the Dual Attention Fusion (DAF). In addition, we propose a dual auxiliary loss to explicitly constrain dual branches, preventing the network from relying only on global semantic information from the deep large receptive field and ignoring the local texture. The proposed RoadMamba achieves the state-of-the-art performance in experiments on a large-scale road surface classification dataset containing 1 million samples.