Feature Attribution Stability Suite: How Stable Are Post-Hoc Attributions?

TL;DR

The paper introduces FASS, a benchmark suite for evaluating the stability of post-hoc feature attribution methods under various realistic perturbations, emphasizing the importance of prediction invariance.

Contribution

FASS provides a comprehensive, prediction-invariance conditioned evaluation framework with multiple stability metrics across diverse perturbations and datasets.

Findings

Geometric perturbations cause more attribution instability than photometric ones.

Without prediction-invariance filtering, up to 99% of attribution pairs involve prediction changes.

Grad-CAM shows the highest stability among evaluated attribution methods.

Abstract

Post-hoc feature attribution methods are widely deployed in safety-critical vision systems, yet their stability under realistic input perturbations remains poorly characterized. Existing metrics evaluate explanations primarily under additive noise, collapse stability to a single scalar, and fail to condition on prediction preservation, conflating explanation fragility with model sensitivity. We introduce the Feature Attribution Stability Suite (FASS), a benchmark that enforces prediction-invariance filtering, decomposes stability into three complementary metrics: structural similarity, rank correlation, and top-k Jaccard overlap-and evaluates across geometric, photometric, and compression perturbations. Evaluating four attribution methods (Integrated Gradients, GradientSHAP, Grad-CAM, LIME) across four architectures and three datasets-ImageNet-1K, MS COCO, and CIFAR-10, FASS shows that stability estimates depend critically on perturbation family and prediction-invariance filtering. Geometric perturbations expose substantially greater attribution instability than photometric changes, and without conditioning on prediction preservation, up to 99% of evaluated pairs involve changed predictions. Under this controlled evaluation, we observe consistent method-level trends, with Grad-CAM achieving the highest stability across datasets.