Areas of Improvement for Autonomous Vehicles: A Machine Learning Analysis of Disengagement Reports

TL;DR

This paper uses machine learning and NLP techniques to analyze disengagement reports of autonomous vehicles, identifying key areas for technological improvement based on clustering and manual categorization of report data.

Contribution

It introduces a novel NLP and clustering approach to analyze disengagement reports, providing insights into common failure modes and areas needing improvement in AV technology.

Findings

Identification of frequent disengagement factors

Clustering of reports reveals common failure patterns

Analysis highlights specific technological weaknesses

Abstract

Since 2014, the California Department of Motor Vehicles (CDMV) has compiled information from manufacturers of autonomous vehicles (AVs) regarding factors that lead to the disengagement from autonomous driving mode in these vehicles. These disengagement reports (DRs) contain information detailing whether the AV disengaged from autonomous mode due to technology failure, manual override, or other factors during driving tests. This paper presents a machine learning (ML) based analysis of the information from the 2023 DRs. We use a natural language processing (NLP) approach to extract important information from the description of a disengagement, and use the k-Means clustering algorithm to group report entries together. The cluster frequency is then analyzed, and each cluster is manually categorized based on the factors leading to disengagement. We discuss findings from previous years' DRs, and provide our own analysis to identify areas of improvement for AVs.