FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods

TL;DR

FunnyBirds introduces a synthetic dataset for evaluating explainable AI methods at a part-based level, enabling automatic, systematic analysis of model explanations through semantic image interventions.

Contribution

The paper presents FunnyBirds, a synthetic dataset with protocols for automatic, part-level evaluation of XAI methods, addressing the lack of ground-truth explanations in the field.

Findings

Part-level explanation analysis reveals strengths and weaknesses of XAI methods.

Automatic evaluation framework enables systematic comparison of 24 model-XAI combinations.

Part removal interventions improve understanding of model decision processes.

Abstract

The field of explainable artificial intelligence (XAI) aims to uncover the inner workings of complex deep neural models. While being crucial for safety-critical domains, XAI inherently lacks ground-truth explanations, making its automatic evaluation an unsolved problem. We address this challenge by proposing a novel synthetic vision dataset, named FunnyBirds, and accompanying automatic evaluation protocols. Our dataset allows performing semantically meaningful image interventions, e.g., removing individual object parts, which has three important implications. First, it enables analyzing explanations on a part level, which is closer to human comprehension than existing methods that evaluate on a pixel level. Second, by comparing the model output for inputs with removed parts, we can estimate ground-truth part importances that should be reflected in the explanations. Third, by mapping individual explanations into a common space of part importances, we can analyze a variety of different explanation types in a single common framework. Using our tools, we report results for 24 different combinations of neural models and XAI methods, demonstrating the strengths and weaknesses of the assessed methods in a fully automatic and systematic manner.