A Backdoor-based Explainable AI Benchmark for High Fidelity Evaluation of Attributions

TL;DR

This paper introduces a novel backdoor-based benchmark for evaluating attribution methods in explainable AI, addressing the challenge of assessing faithfulness without ground truth and providing a standardized, fair evaluation framework.

Contribution

It proposes a new benchmark (BackX) that meets fidelity criteria, establishes its theoretical advantages, and offers a standardized setup for fair comparison of attribution methods.

Findings

BackX outperforms existing benchmarks in attribution evaluation.

The standardized setup reduces confounding factors in benchmarking.

Insights into neural Trojan defense using attribution methods.

Abstract

Attribution methods compute importance scores for input features to explain model predictions. However, assessing the faithfulness of these methods remains challenging due to the absence of attribution ground truth to model predictions. In this work, we first identify a set of fidelity criteria that reliable benchmarks for attribution methods are expected to fulfill, thereby facilitating a systematic assessment of attribution benchmarks. Next, we introduce a Backdoor-based eXplainable AI benchmark (BackX) that adheres to the desired fidelity criteria. We theoretically establish the superiority of our approach over the existing benchmarks for well-founded attribution evaluation. With extensive analysis, we further establish a standardized evaluation setup that mitigates confounding factors such as post-processing techniques and explained predictions, thereby ensuring a fair and consistent benchmarking. This setup is ultimately employed for a comprehensive comparison of existing methods using BackX. Finally, our analysis also offers insights into defending against neural Trojans by utilizing the attributions.