Reliable Explanations or Random Noise? A Reliability Metric for XAI

TL;DR

This paper introduces the Explanation Reliability Index (ERI), a set of metrics to evaluate the stability of explanations in XAI, revealing widespread unreliability in popular methods like SHAP and IG under realistic conditions.

Contribution

The paper proposes ERI, a novel framework with formal guarantees and benchmarks for assessing explanation stability in XAI, addressing a critical gap in explanation reliability measurement.

Findings

Popular explanation methods often produce unstable explanations.

ERI metrics reveal significant reliability issues in current XAI techniques.

The benchmark enables systematic testing of explanation robustness.

Abstract

In recent years, explaining decisions made by complex machine learning models has become essential in high-stakes domains such as energy systems, healthcare, finance, and autonomous systems. However, the reliability of these explanations, namely, whether they remain stable and consistent under realistic, non-adversarial changes, remains largely unmeasured. Widely used methods such as SHAP and Integrated Gradients (IG) are well-motivated by axiomatic notions of attribution, yet their explanations can vary substantially even under system-level conditions, including small input perturbations, correlated representations, and minor model updates. Such variability undermines explanation reliability, as reliable explanations should remain consistent across equivalent input representations and small, performance-preserving model changes. We introduce the Explanation Reliability Index (ERI), a family of metrics that quantifies explanation stability under four reliability axioms: robustness to small input perturbations, consistency under feature redundancy, smoothness across model evolution, and resilience to mild distributional shifts. For each axiom, we derive formal guarantees, including Lipschitz-type bounds and temporal stability results. We further propose ERI-T, a dedicated measure of temporal reliability for sequential models, and introduce ERI-Bench, a benchmark designed to systematically stress-test explanation reliability across synthetic and real-world datasets. Experimental results reveal widespread reliability failures in popular explanation methods, showing that explanations can be unstable under realistic deployment conditions. By exposing and quantifying these instabilities, ERI enables principled assessment of explanation reliability and supports more trustworthy explainable AI (XAI) systems.