Evaluating Model Explanations without Ground Truth

TL;DR

This paper introduces AXE, a novel framework for evaluating local feature importance explanations without needing ground-truth explanations, addressing limitations of existing methods and enabling fair comparison of explanations.

Contribution

The paper proposes AXE, a ground-truth agnostic evaluation framework for explanations that does not rely on model sensitivity or ideal explanations, advancing explanation assessment methods.

Findings

AXE provides an independent measure of explanation quality.

AXE can detect explanation fairwashing.

AXE correlates well with baseline methods.

Abstract

There can be many competing and contradictory explanations for a single model prediction, making it difficult to select which one to use. Current explanation evaluation frameworks measure quality by comparing against ideal "ground-truth" explanations, or by verifying model sensitivity to important inputs. We outline the limitations of these approaches, and propose three desirable principles to ground the future development of explanation evaluation strategies for local feature importance explanations. We propose a ground-truth Agnostic eXplanation Evaluation framework (AXE) for evaluating and comparing model explanations that satisfies these principles. Unlike prior approaches, AXE does not require access to ideal ground-truth explanations for comparison, or rely on model sensitivity - providing an independent measure of explanation quality. We verify AXE by comparing with baselines, and show how it can be used to detect explanation fairwashing. Our code is available at https://github.com/KaiRawal/Evaluating-Model-Explanations-without-Ground-Truth.