Learning Multimodal Embeddings for Traffic Accident Prediction and Causal Estimation

TL;DR

This paper develops a multimodal dataset combining road network data and satellite images to improve traffic accident prediction and causal analysis, demonstrating significant accuracy gains and environmental impact insights.

Contribution

It introduces a large, annotated multimodal dataset and evaluates methods that integrate visual and network data for accident prediction and causal inference.

Findings

Integrating visual and network data improves accident prediction accuracy by 3.7%.

Satellite imagery features are crucial for accurate accident prediction.

Higher precipitation, speed, and seasonal factors significantly increase accident rates.

Abstract

We consider analyzing traffic accident patterns using both road network data and satellite images aligned to road graph nodes. Previous work for predicting accident occurrences relies primarily on road network structural features while overlooking physical and environmental information from the road surface and its surroundings. In this work, we construct a large multimodal dataset spanning six U.S. states, containing nine million traffic accident records from official sources, and one million high-resolution satellite images for each node of the road network. Additionally, every node is annotated with features such as the region's weather statistics and road type (e.g., residential vs. motorway), and each edge is annotated with traffic volume information (i.e., Average Annual Daily Traffic). Utilizing this dataset, we conduct a comprehensive evaluation of multimodal learning methods that integrate both visual and network embeddings. Our findings show that integrating both data modalities improves prediction accuracy, achieving an average AUROC of $90.1\%$, a $3.7\%$ gain over graph neural network models that use only graph structures. With the improved embeddings, we conduct a causal analysis using a matching estimator to identify the key factors influencing traffic accidents. We find that accident rates rise by $24\%$ under higher precipitation, by $22\%$ on higher-speed roads such as motorways, and by $29\%$ due to seasonal patterns, after adjusting for other confounding factors. Ablation studies confirm that satellite imagery features are essential for achieving accurate prediction.