Counterfactual Gradients-based Quantification of Prediction Trust in Neural Networks

TL;DR

GradTrust is a novel trust measure for neural networks that uses counterfactual gradient variance to detect mispredictions, outperforming existing methods across multiple architectures and datasets.

Contribution

The paper introduces GradTrust, a new method leveraging counterfactual gradient variance for quantifying neural network prediction trust at inference.

Findings

GradTrust outperforms existing techniques in misprediction detection on ImageNet.

Simple classifiers like negative log likelihood outperform state-of-the-art uncertainty methods.

GradTrust ranks among the top methods in most experimental settings.

Abstract

The widespread adoption of deep neural networks in machine learning calls for an objective quantification of esoteric trust. In this paper we propose GradTrust, a classification trust measure for large-scale neural networks at inference. The proposed method utilizes variance of counterfactual gradients, i.e. the required changes in the network parameters if the label were different. We show that GradTrust is superior to existing techniques for detecting misprediction rates on $50000$ images from ImageNet validation dataset. Depending on the network, GradTrust detects images where either the ground truth is incorrect or ambiguous, or the classes are co-occurring. We extend GradTrust to Video Action Recognition on Kinetics-400 dataset. We showcase results on $14$ architectures pretrained on ImageNet and $5$ architectures pretrained on Kinetics-400. We observe the following: (i) simple methodologies like negative log likelihood and margin classifiers outperform state-of-the-art uncertainty and out-of-distribution detection techniques for misprediction rates, and (ii) the proposed GradTrust is in the Top-2 performing methods on $37$ of the considered $38$ experimental modalities. The code is available at: https://github.com/olivesgatech/GradTrust