SalM$^{2}$: An Extremely Lightweight Saliency Mamba Model for Real-Time Cognitive Awareness of Driver Attention

TL;DR

This paper introduces SalM$^{2}$, a highly lightweight real-time model for driver attention recognition that uses minimal parameters while maintaining near state-of-the-art accuracy in complex traffic scenes.

Contribution

The paper presents a novel, extremely lightweight saliency model based on the Mamba framework, optimized for real-time driver attention recognition with significantly fewer parameters.

Findings

Uses only 0.08M parameters, about 0.09-11.16% of other models.

Achieves over 98% of the performance of state-of-the-art models.

Maintains high accuracy in complex traffic scene scenarios.

Abstract

Driver attention recognition in driving scenarios is a popular direction in traffic scene perception technology. It aims to understand human driver attention to focus on specific targets/objects in the driving scene. However, traffic scenes contain not only a large amount of visual information but also semantic information related to driving tasks. Existing methods lack attention to the actual semantic information present in driving scenes. Additionally, the traffic scene is a complex and dynamic process that requires constant attention to objects related to the current driving task. Existing models, influenced by their foundational frameworks, tend to have large parameter counts and complex structures. Therefore, this paper proposes a real-time saliency Mamba network based on the latest Mamba framework. As shown in Figure 1, our model uses very few parameters (0.08M, only 0.09~11.16% of other models), while maintaining SOTA performance or achieving over 98% of the SOTA model's performance.