Improving Prediction Confidence in Learning-Enabled Autonomous Systems

TL;DR

This paper introduces a real-time feedback loop using a conformal prediction classifier with a triplet network to improve confidence and accuracy in autonomous system predictions, demonstrated on traffic sign recognition.

Contribution

It presents a novel feedback mechanism combined with a conformal prediction classifier based on triplet networks for enhanced prediction confidence in autonomous systems.

Findings

Reduced error rate in traffic sign recognition

Efficient and scalable to high-dimensional data

Real-time feedback loop improves prediction accuracy

Abstract

Autonomous systems use extensively learning-enabled components such as deep neural networks (DNNs) for prediction and decision making. In this paper, we utilize a feedback loop between learning-enabled components used for classification and the sensors of an autonomous system in order to improve the confidence of the predictions. We design a classifier using Inductive Conformal Prediction (ICP) based on a triplet network architecture in order to learn representations that can be used to quantify the similarity between test and training examples. The method allows computing confident set predictions with an error rate predefined using a selected significance level. A feedback loop that queries the sensors for a new input is used to further refine the predictions and increase the classification accuracy. The method is computationally efficient, scalable to high-dimensional inputs, and can be executed in a feedback loop with the system in real-time. The approach is evaluated using a traffic sign recognition dataset and the results show that the error rate is reduced.