P-TAME: Explain Any Image Classifier with Trained Perturbations

TL;DR

P-TAME is a novel, efficient, model-agnostic explanation method for image classifiers that uses trained perturbations and an auxiliary classifier to generate high-resolution explanations in a single inference pass.

Contribution

It introduces P-TAME, a new perturbation-based explanation approach that is efficient, model-agnostic, and capable of producing high-resolution explanations during inference.

Findings

P-TAME matches or outperforms existing explainability methods.

It provides high-resolution explanations with a single forward pass.

The method is effective across different classifier architectures.

Abstract

The adoption of Deep Neural Networks (DNNs) in critical fields where predictions need to be accompanied by justifications is hindered by their inherent black-box nature. In this paper, we introduce P-TAME (Perturbation-based Trainable Attention Mechanism for Explanations), a model-agnostic method for explaining DNN-based image classifiers. P-TAME employs an auxiliary image classifier to extract features from the input image, bypassing the need to tailor the explanation method to the internal architecture of the backbone classifier being explained. Unlike traditional perturbation-based methods, which have high computational requirements, P-TAME offers an efficient alternative by generating high-resolution explanations in a single forward pass during inference. We apply P-TAME to explain the decisions of VGG-16, ResNet-50, and ViT-B-16, three distinct and widely used image classifiers. Quantitative and qualitative results show that our method matches or outperforms previous explainability methods, including model-specific approaches. Code and trained models will be released upon acceptance.