Decoupling Pixel Flipping and Occlusion Strategy for Consistent XAI Benchmarks

TL;DR

This paper introduces a new framework for evaluating occlusion-based XAI methods by measuring their reliability with the R-OMS score and combining influence measures into the SRG metric, ensuring consistent rankings across strategies.

Contribution

It proposes the R-OMS score for assessing occlusion strategy reliability and the SRG measure to unify influence rankings, resolving inconsistencies in pixel flipping benchmarks.

Findings

R-OMS score effectively compares occlusion strategies.

SRG measure provides consistent feature importance rankings.

Validated on 40 different occlusion strategies.

Abstract

Feature removal is a central building block for eXplainable AI (XAI), both for occlusion-based explanations (Shapley values) as well as their evaluation (pixel flipping, PF). However, occlusion strategies can vary significantly from simple mean replacement up to inpainting with state-of-the-art diffusion models. This ambiguity limits the usefulness of occlusion-based approaches. For example, PF benchmarks lead to contradicting rankings. This is amplified by competing PF measures: Features are either removed starting with most influential first (MIF) or least influential first (LIF). This study proposes two complementary perspectives to resolve this disagreement problem. Firstly, we address the common criticism of occlusion-based XAI, that artificial samples lead to unreliable model evaluations. We propose to measure the reliability by the R(eference)-Out-of-Model-Scope (OMS) score. The R-OMS score enables a systematic comparison of occlusion strategies and resolves the disagreement problem by grouping consistent PF rankings. Secondly, we show that the insightfulness of MIF and LIF is conversely dependent on the R-OMS score. To leverage this, we combine the MIF and LIF measures into the symmetric relevance gain (SRG) measure. This breaks the inherent connection to the underlying occlusion strategy and leads to consistent rankings. This resolves the disagreement problem, which we verify for a set of 40 different occlusion strategies.