AccidentGPT: Large Multi-Modal Foundation Model for Traffic Accident Analysis

TL;DR

AccidentGPT is a multi-modal foundation model designed for automatic, objective, and privacy-preserving traffic accident analysis, capable of reconstructing accident videos and performing multi-task analysis using diverse data inputs.

Contribution

The paper introduces AccidentGPT, a novel multi-modal foundation model that integrates multi-modal data, feedback prompts, and a hybrid training schema for comprehensive traffic accident analysis.

Findings

Reconstructs accident process videos with detailed dynamics

Performs multi-task analysis with multi-modal outputs

Leverages labeled and unlabeled data effectively

Abstract

Traffic accident analysis is pivotal for enhancing public safety and developing road regulations. Traditional approaches, although widely used, are often constrained by manual analysis processes, subjective decisions, uni-modal outputs, as well as privacy issues related to sensitive data. This paper introduces the idea of AccidentGPT, a foundation model of traffic accident analysis, which incorporates multi-modal input data to automatically reconstruct the accident process video with dynamics details, and furthermore provide multi-task analysis with multi-modal outputs. The design of the AccidentGPT is empowered with a multi-modality prompt with feedback for task-oriented adaptability, a hybrid training schema to leverage labelled and unlabelled data, and a edge-cloud split configuration for data privacy. To fully realize the functionalities of this model, we proposes several research opportunities. This paper serves as the stepping stone to fill the gaps in traditional approaches of traffic accident analysis and attract the research community attention for automatic, objective, and privacy-preserving traffic accident analysis.