Reliable Probability Intervals For Classification Using Inductive Venn Predictors Based on Distance Learning

TL;DR

This paper introduces a real-time, efficient method using Inductive Venn Predictors and distance metric learning to compute reliable probability intervals for neural network predictions, enhancing confidence calibration in high-dimensional data applications.

Contribution

It proposes a novel approach combining Inductive Venn Predictors with distance metric learning for accurate, real-time confidence estimation in neural network classification tasks.

Findings

Improved accuracy and calibration in image classification.

Effective detection of IoT botnet attacks.

Method is computationally efficient for real-time use.

Abstract

Deep neural networks are frequently used by autonomous systems for their ability to learn complex, non-linear data patterns and make accurate predictions in dynamic environments. However, their use as black boxes introduces risks as the confidence in each prediction is unknown. Different frameworks have been proposed to compute accurate confidence measures along with the predictions but at the same time introduce a number of limitations like execution time overhead or inability to be used with high-dimensional data. In this paper, we use the Inductive Venn Predictors framework for computing probability intervals regarding the correctness of each prediction in real-time. We propose taxonomies based on distance metric learning to compute informative probability intervals in applications involving high-dimensional inputs. Empirical evaluation on image classification and botnet attacks detection in Internet-of-Things (IoT) applications demonstrates improved accuracy and calibration. The proposed method is computationally efficient, and therefore, can be used in real-time.