EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding

TL;DR

EyeCue is a novel framework that detects driver cognitive distraction by analyzing the interaction between eye gaze and egocentric video, supported by a new multi-scenario dataset, CogDrive.

Contribution

The paper introduces EyeCue, a gaze-empowered video understanding model, and CogDrive, a comprehensive dataset, advancing cognitive distraction detection in driving scenarios.

Findings

EyeCue achieves 74.38% accuracy, outperforming baselines by over 7%.

It maintains over 70% accuracy across diverse scenarios.

Modeling gaze-context interactions enhances detection performance.

Abstract

Driver cognitive distraction is a major cause of road collisions and remains difficult to detect. Unlike manual or visual distraction, cognitive distraction is diverted by thoughts unrelated to driving, even when the driver appears visually attentive and exhibits no explicit physical movements. In this work, we propose EyeCue, a gaze-empowered egocentric video understanding framework, to detect driver cognitive distraction. A key insight is that cognitive distraction manifests in the interaction between eye gaze and visual context. To capture this interaction, EyeCue integrates eye gaze with egocentric video to enable context-aware modeling of the driver's attention over time. Furthermore, to tackle the limited scale and diversity of existing datasets, we introduce CogDrive, a comprehensive multi-scenario dataset that augments four existing driving datasets with cognitive distraction annotations. Through extensive evaluations on CogDrive, we show that EyeCue achieves the highest accuracy of 74.38%, outperforming 11 baselines from 6 model families by over 7%. Notably, EyeCue can achieve an accuracy of over 70% across various driving scenarios (different road types, times of day, and weather conditions) with strong generalizability. These results highlight the importance of modeling gaze-context interactions and the effectiveness of cross-modal interaction modeling for multimodal cognitive distraction detection. Our codes and CogDrive dataset resources are available at https://github.com/langzhang2000/EyeCue.