Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL)

TL;DR

This study evaluates machine learning and deep learning models, especially the Albert Model, for identifying misclassified crash narratives, demonstrating that hybrid automated-expert approaches significantly improve data accuracy in transportation safety analysis.

Contribution

It introduces a comprehensive comparison of ML and DL models for crash narrative classification and highlights the effectiveness of the Albert Model in aligning with expert judgments.

Findings

Albert Model achieved 73% agreement with experts.

Hybrid approach reduced error rates by 54.2%.

Traditional ML outperformed some DL methods in accuracy.

Abstract

This research investigates the efficacy of machine learning (ML) and deep learning (DL) methods in detecting misclassified intersection-related crashes in police-reported narratives. Using 2019 crash data from the Iowa Department of Transportation, we implemented and compared a comprehensive set of models, including Support Vector Machine (SVM), XGBoost, BERT Sentence Embeddings, BERT Word Embeddings, and Albert Model. Model performance was systematically validated against expert reviews of potentially misclassified narratives, providing a rigorous assessment of classification accuracy. Results demonstrated that while traditional ML methods exhibited superior overall performance compared to some DL approaches, the Albert Model achieved the highest agreement with expert classifications (73% with Expert 1) and original tabular data (58%). Statistical analysis revealed that the Albert Model maintained performance levels similar to inter-expert consistency rates, significantly outperforming other approaches, particularly on ambiguous narratives. This work addresses a critical gap in transportation safety research through multi-modal integration analysis, which achieved a 54.2% reduction in error rates by combining narrative text with structured crash data. We conclude that hybrid approaches combining automated classification with targeted expert review offer a practical methodology for improving crash data quality, with substantial implications for transportation safety management and policy development.