Physical Depth-aware Early Accident Anticipation: A Multi-dimensional Visual Feature Fusion Framework

TL;DR

This paper presents a depth-aware multi-dimensional visual feature fusion framework for early accident anticipation in dashcam videos, improving safety by capturing 3D spatial relationships and occlusion handling.

Contribution

It introduces a novel depth-aware learning framework that integrates monocular depth features, visual interaction, and dynamic features for better accident prediction.

Findings

Achieves state-of-the-art performance on public datasets.

Effectively incorporates 3D depth information for early accident detection.

Mitigates occlusion issues with a reconstruction adjacency matrix.

Abstract

Early accident anticipation from dashcam videos is a highly desirable yet challenging task for improving the safety of intelligent vehicles. Existing advanced accident anticipation approaches commonly model the interaction among traffic agents (e.g., vehicles, pedestrians, etc.) in the coarse 2D image space, which may not adequately capture their true positions and interactions. To address this limitation, we propose a physical depth-aware learning framework that incorporates the monocular depth features generated by a large model named Depth-Anything to introduce more fine-grained spatial 3D information. Furthermore, the proposed framework also integrates visual interaction features and visual dynamic features from traffic scenes to provide a more comprehensive perception towards the scenes. Based on these multi-dimensional visual features, the framework captures early indicators of accidents through the analysis of interaction relationships between objects in sequential frames. Additionally, the proposed framework introduces a reconstruction adjacency matrix for key traffic participants that are occluded, mitigating the impact of occluded objects on graph learning and maintaining the spatio-temporal continuity. Experimental results on public datasets show that the proposed framework attains state-of-the-art performance, highlighting the effectiveness of incorporating visual depth features and the superiority of the proposed framework.