# Online Black-Box Confidence Estimation of Deep Neural Networks

## TL;DR

This paper introduces a black-box confidence estimation method called neighborhood confidence (NHC) for deep neural networks, improving real-time reliability assessment in autonomous driving systems without needing access to internal model details.

## Contribution

The paper proposes NHC, a novel black-box confidence metric for DNNs that works in real-time and does not require access to gradients or training data, suitable for safety-critical applications.

## Key findings

- NHC performs better or on par with white-box methods in various data shift scenarios.
- NHC is effective with limited data regimes, enabling real-time confidence estimation.
- Evaluation includes out-of-domain, adversarial, and distribution-shifted data.

## Abstract

Autonomous driving (AD) and advanced driver assistance systems (ADAS) increasingly utilize deep neural networks (DNNs) for improved perception or planning. Nevertheless, DNNs are quite brittle when the data distribution during inference deviates from the data distribution during training. This represents a challenge when deploying in partly unknown environments like in the case of ADAS. At the same time, the standard confidence of DNNs remains high even if the classification reliability decreases. This is problematic since following motion control algorithms consider the apparently confident prediction as reliable even though it might be considerably wrong. To reduce this problem real-time capable confidence estimation is required that better aligns with the actual reliability of the DNN classification. Additionally, the need exists for black-box confidence estimation to enable the homogeneous inclusion of externally developed components to an entire system. In this work we explore this use case and introduce the neighborhood confidence (NHC) which estimates the confidence of an arbitrary DNN for classification. The metric can be used for black-box systems since only the top-1 class output is required and does not need access to the gradients, the training dataset or a hold-out validation dataset. Evaluation on different data distributions, including small in-domain distribution shifts, out-of-domain data or adversarial attacks, shows that the NHC performs better or on par with a comparable method for online white-box confidence estimation in low data regimes which is required for real-time capable AD/ADAS.