A Unified Framework for Evaluating and Enhancing the Transparency of Explainable AI Methods via Perturbation-Gradient Consensus Attribution

TL;DR

This paper introduces a comprehensive evaluation framework for XAI methods and proposes PGCA, a novel attribution technique that combines perturbation importance with gradient-based methods, demonstrating superior performance across multiple domains.

Contribution

The paper presents a unified multi-criteria evaluation framework and a new attribution method, PGCA, enhancing fidelity, interpretability, and fairness in explainable AI.

Findings

PGCA outperforms baselines in fidelity, interpretability, and fairness metrics.

The evaluation framework effectively ranks XAI methods across diverse domains.

Code and results are publicly available for reproducibility.

Abstract

Explainable Artificial Intelligence (XAI) methods are increasingly used in safety-critical domains, yet there is no unified framework to jointly evaluate fidelity, interpretability, robustness, fairness, and completeness. We address this gap through two contributions. First, we propose a multi-criteria evaluation framework that formalizes these five criteria using principled metrics: fidelity via prediction-gap analysis; interpretability via a composite concentration-coherence-contrast score; robustness via cosine-similarity perturbation stability; fairness via Jensen-Shannon divergence across demographic groups; and completeness via feature-ablation coverage. These are integrated using an entropy-weighted dynamic scoring scheme that adapts to domain-specific priorities. Second, we introduce Perturbation-Gradient Consensus Attribution (PGCA), which fuses grid-based perturbation importance with Grad-CAM++ through consensus amplification and adaptive contrast enhancement, combining perturbation fidelity with gradient-based spatial precision. We evaluate across five domains (brain tumor MRI, plant disease, security screening, gender, and sunglass detection) using fine-tuned ResNet-50 models. PGCA achieves the best performance in fidelity $(2.22 \pm 1.62)$, interpretability $(3.89 \pm 0.33)$, and fairness $(4.95 \pm 0.03)$, with statistically significant improvements over baselines $(p < 10^{-7})$. Sensitivity analysis shows stable rankings (Kendall's $(\tau \geq 0.88)$). Code and results are publicly available.