SCAN: Visual Explanations with Self-Confidence and Analysis Networks

TL;DR

SCAN introduces a universal, autoencoder-based framework for visual explanations in AI, providing high-fidelity, clear, and architecture-agnostic insights into deep learning models' decision processes.

Contribution

It proposes a novel, architecture-agnostic explanation method using an AutoEncoder and Information Bottleneck, improving clarity and fidelity over existing approaches.

Findings

SCAN outperforms existing methods on multiple quantitative metrics.

Produces clearer, object-focused explanations.

Works effectively across CNNs and Transformers.

Abstract

Explainable AI (XAI) has become essential in computer vision to make the decision-making processes of deep learning models transparent. However, current visual explanation (XAI) methods face a critical trade-off between the high fidelity of architecture-specific methods and the broad applicability of universal ones. This often results in abstract or fragmented explanations and makes it difficult to compare explanatory power across diverse model families, such as CNNs and Transformers. This paper introduces the Self-Confidence and Analysis Networks (SCAN), a novel universal framework that overcomes these limitations for both convolutional neural network and transformer architectures. SCAN utilizes an AutoEncoder-based approach to reconstruct features from a model's intermediate layers. Guided by the Information Bottleneck principle, it generates a high-resolution Self-Confidence Map that identifies information-rich regions. Extensive experiments on diverse architectures and datasets demonstrate that SCAN consistently achieves outstanding performance on various quantitative metrics such as AUC-D, Negative AUC, Drop%, and Win%. Qualitatively, it produces significantly clearer, object-focused explanations than existing methods. By providing a unified framework that is both architecturally universal and highly faithful, SCAN enhances model transparency and offers a more reliable tool for understanding the decision-making processes of complex neural networks.