AHMF: Adaptive Hybrid-Memory-Fusion Model for Driver Attention Prediction

TL;DR

This paper introduces AHMF, a driver attention prediction model that mimics human memory processes by combining working and long-term memory, leading to more accurate and human-like predictions in traffic scenes.

Contribution

The paper proposes a novel hybrid memory fusion model that explicitly integrates working and long-term memory for driver attention prediction, inspired by cognitive science.

Findings

Significant performance improvements on multiple datasets.

Effective integration of hybrid memories enhances prediction accuracy.

Parallel training across datasets enriches long-term memory.

Abstract

Accurate driver attention prediction can serve as a critical reference for intelligent vehicles in understanding traffic scenes and making informed driving decisions. Though existing studies on driver attention prediction improved performance by incorporating advanced saliency detection techniques, they overlooked the opportunity to achieve human-inspired prediction by analyzing driving tasks from a cognitive science perspective. During driving, drivers' working memory and long-term memory play crucial roles in scene comprehension and experience retrieval, respectively. Together, they form situational awareness, facilitating drivers to quickly understand the current traffic situation and make optimal decisions based on past driving experiences. To explicitly integrate these two types of memory, this paper proposes an Adaptive Hybrid-Memory-Fusion (AHMF) driver attention prediction model to achieve more human-like predictions. Specifically, the model first encodes information about specific hazardous stimuli in the current scene to form working memories. Then, it adaptively retrieves similar situational experiences from the long-term memory for final prediction. Utilizing domain adaptation techniques, the model performs parallel training across multiple datasets, thereby enriching the accumulated driving experience within the long-term memory module. Compared to existing models, our model demonstrates significant improvements across various metrics on multiple public datasets, proving the effectiveness of integrating hybrid memories in driver attention prediction.