Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review

TL;DR

This paper reviews deep learning techniques for traffic video analysis, focusing on safety metrics, processing frameworks, and open challenges to guide future research and practical applications in traffic safety and autonomous driving.

Contribution

It provides a comprehensive comparison of DL-based algorithms for traffic video processing, offers a modular framework, and discusses open-source tools, datasets, and real-world applications.

Findings

Comparative analysis of conventional and DL-based algorithms for each processing step.

Identification of gaps in current traffic safety analysis methods.

Review of commercial traffic monitoring systems and future challenges.

Abstract

This paper explores Deep Learning (DL) methods that are used or have the potential to be used for traffic video analysis, emphasizing driving safety for both Autonomous Vehicles (AVs) and human-operated vehicles. We present a typical processing pipeline, which can be used to understand and interpret traffic videos by extracting operational safety metrics and providing general hints and guidelines to improve traffic safety. This processing framework includes several steps, including video enhancement, video stabilization, semantic and incident segmentation, object detection and classification, trajectory extraction, speed estimation, event analysis, modeling and anomaly detection. Our main goal is to guide traffic analysts to develop their own custom-built processing frameworks by selecting the best choices for each step and offering new designs for the lacking modules by providing a comparative analysis of the most successful conventional and DL-based algorithms proposed for each step. We also review existing open-source tools and public datasets that can help train DL models. To be more specific, we review exemplary traffic problems and mentioned requires steps for each problem. Besides, we investigate connections to the closely related research areas of drivers' cognition evaluation, Crowd-sourcing-based monitoring systems, Edge Computing in roadside infrastructures, Automated Driving Systems (ADS)-equipped vehicles, and highlight the missing gaps. Finally, we review commercial implementations of traffic monitoring systems, their future outlook, and open problems and remaining challenges for widespread use of such systems.