ClaudesLens: Uncertainty Quantification in Computer Vision Models

TL;DR

This paper introduces a theoretical framework using Shannon entropy to quantify uncertainty in computer vision models, enhancing understanding of model confidence in classification tasks.

Contribution

The paper proposes a novel method to measure uncertainty in neural network outputs through entropy-based perturbations and evaluation metrics, applicable across various AI applications.

Findings

Entropy perturbation reveals uncertainty levels in models.

Proposed metrics effectively evaluate model confidence.

Framework applicable to different neural network architectures.

Abstract

In a world where more decisions are made using artificial intelligence, it is of utmost importance to ensure these decisions are well-grounded. Neural networks are the modern building blocks for artificial intelligence. Modern neural network-based computer vision models are often used for object classification tasks. Correctly classifying objects with \textit{certainty} has become of great importance in recent times. However, quantifying the inherent \textit{uncertainty} of the output from neural networks is a challenging task. Here we show a possible method to quantify and evaluate the uncertainty of the output of different computer vision models based on Shannon entropy. By adding perturbation of different levels, on different parts, ranging from the input to the parameters of the network, one introduces entropy to the system. By quantifying and evaluating the perturbed models on the proposed PI and PSI metrics, we can conclude that our theoretical framework can grant insight into the uncertainty of predictions of computer vision models. We believe that this theoretical framework can be applied to different applications for neural networks. We believe that Shannon entropy may eventually have a bigger role in the SOTA (State-of-the-art) methods to quantify uncertainty in artificial intelligence. One day we might be able to apply Shannon entropy to our neural systems.