AVD2: Accident Video Diffusion for Accident Video Description

TL;DR

This paper introduces AVD2, a framework that generates accident videos with detailed descriptions to improve understanding and prevention of traffic accidents, supported by a new dataset and state-of-the-art results.

Contribution

The work presents a novel accident video diffusion framework and the EMM-AU dataset, advancing accident scene understanding and analysis.

Findings

Achieved state-of-the-art performance on accident video understanding metrics.

Demonstrated improved accident analysis and prevention capabilities.

Provided a new dataset for accident video description and reasoning.

Abstract

Traffic accidents present complex challenges for autonomous driving, often featuring unpredictable scenarios that hinder accurate system interpretation and responses. Nonetheless, prevailing methodologies fall short in elucidating the causes of accidents and proposing preventive measures due to the paucity of training data specific to accident scenarios. In this work, we introduce AVD2 (Accident Video Diffusion for Accident Video Description), a novel framework that enhances accident scene understanding by generating accident videos that aligned with detailed natural language descriptions and reasoning, resulting in the contributed EMM-AU (Enhanced Multi-Modal Accident Video Understanding) dataset. Empirical results reveal that the integration of the EMM-AU dataset establishes state-of-the-art performance across both automated metrics and human evaluations, markedly advancing the domains of accident analysis and prevention. Project resources are available at https://an-answer-tree.github.io