Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control

TL;DR

This paper investigates how real-time uncertainty measures from deep neural networks can predict imminent crashes in autonomous driving, enhancing safety by providing early warnings.

Contribution

It introduces evaluation techniques for uncertainty in end-to-end self-driving models and demonstrates mutual information as a promising crash predictor.

Findings

Mutual information correlates with upcoming crashes.

Uncertainty measures can predict crashes up to five seconds in advance.

The approach works across different neural network architectures.

Abstract

A rise in popularity of Deep Neural Networks (DNNs), attributed to more powerful GPUs and widely available datasets, has seen them being increasingly used within safety-critical domains. One such domain, self-driving, has benefited from significant performance improvements, with millions of miles having been driven with no human intervention. Despite this, crashes and erroneous behaviours still occur, in part due to the complexity of verifying the correctness of DNNs and a lack of safety guarantees. In this paper, we demonstrate how quantitative measures of uncertainty can be extracted in real-time, and their quality evaluated in end-to-end controllers for self-driving cars. To this end we utilise a recent method for gathering approximate uncertainty information from DNNs without changing the network's architecture. We propose evaluation techniques for the uncertainty on two separate architectures which use the uncertainty to predict crashes up to five seconds in advance. We find that mutual information, a measure of uncertainty in classification networks, is a promising indicator of forthcoming crashes.