When Stability meets Sufficiency: Informative Explanations that do not Overwhelm

TL;DR

This paper introduces PSEM, a novel explanation method that provides stable, sufficient, and understandable feature attributions by tracing a sequence from the original input to minimal sufficiency, improving interpretability across multiple data modalities.

Contribution

The paper proposes PSEM, a new path-based explanation technique that balances stability, fidelity, and comprehensibility, addressing limitations of minimal sufficiency explanations.

Findings

PSEM produces more stable and understandable explanations.

Experiments show PSEM's effectiveness across image, tabular, and text data.

User study confirms improved comprehension of model behavior.

Abstract

Recent studies evaluating various criteria for explainable artificial intelligence (XAI) suggest that fidelity, stability, and comprehensibility are among the most important metrics considered by users of AI across a diverse collection of usage contexts. We consider these criteria as applied to feature-based attribution methods, which are amongst the most prevalent in XAI literature. Going beyond standard correlation, methods have been proposed that highlight what should be minimally sufficient to justify the classification of an input (viz. pertinent positives). While minimal sufficiency is an attractive property akin to comprehensibility, the resulting explanations are often too sparse for a human to understand and evaluate the local behavior of the model. To overcome these limitations, we incorporate the criteria of stability and fidelity and propose a novel method called Path-Sufficient Explanations Method (PSEM) that outputs a sequence of stable and sufficient explanations for a given input of strictly decreasing size (or value) -- from original input to a minimally sufficient explanation -- which can be thought to trace the local boundary of the model in a stable manner, thus providing better intuition about the local model behavior for the specific input. We validate these claims, both qualitatively and quantitatively, with experiments that show the benefit of PSEM across three modalities (image, tabular and text) as well as versus other path explanations. A user study depicts the strength of the method in communicating the local behavior, where (many) users are able to correctly determine the prediction made by a model.