Multi-Layer Confidence Scoring for Detection of Out-of-Distribution Samples, Adversarial Attacks, and In-Distribution Misclassifications

TL;DR

This paper introduces MACS, a post-hoc framework that analyzes intermediate neural network activations to estimate confidence, detect out-of-distribution samples, adversarial attacks, and misclassifications, outperforming existing methods with lower computational costs.

Contribution

The paper presents MACS, a novel unified post-hoc approach that leverages intermediate activations for confidence scoring and detection of distributional shifts and adversarial attacks.

Findings

MACS surpasses state-of-the-art in detection accuracy.

MACS requires less computational overhead.

Effective on VGG16 and ViTb16 models.

Abstract

The recent explosive growth in Deep Neural Networks applications raises concerns about the black-box usage of such models, with limited trasparency and trustworthiness in high-stakes domains, which have been crystallized as regulatory requirements such as the European Union Artificial Intelligence Act. While models with embedded confidence metrics have been proposed, such approaches cannot be applied to already existing models without retraining, limiting their broad application. On the other hand, post-hoc methods, which evaluate pre-trained models, focus on solving problems related to improving the confidence in the model's predictions, and detecting Out-Of-Distribution or Adversarial Attacks samples as independent applications. To tackle the limited applicability of already existing methods, we introduce Multi-Layer Analysis for Confidence Scoring (MACS), a unified post-hoc framework that analyzes intermediate activations to produce classification-maps. From the classification-maps, we derive a score applicable for confidence estimation, detecting distributional shifts and adversarial attacks, unifying the three problems in a common framework, and achieving performances that surpass the state-of-the-art approaches in our experiments with the VGG16 and ViTb16 models with a fraction of their computational overhead.