The Shape of Explanations: A Topological Account of Rule-Based Explanations in Machine Learning

TL;DR

This paper introduces a topological framework to analyze rule-based explanations in machine learning, clarifying when and why these methods are effective based on the classifier's definability and user knowledge.

Contribution

It proposes a novel topological approach to characterize explainability, linking explanation schemes to user knowledge and domain properties.

Findings

Explainability depends on classifier definability within a topological space.

Different explanation schemes are suitable depending on user knowledge and domain complexity.

The framework unifies various rule-based explanation methods under a common topological perspective.

Abstract

Rule-based explanations provide simple reasons explaining the behavior of machine learning classifiers at given points in the feature space. Several recent methods (Anchors, LORE, etc.) purport to generate rule-based explanations for arbitrary or black-box classifiers. But what makes these methods work in general? We introduce a topological framework for rule-based explanation methods and provide a characterization of explainability in terms of the definability of a classifier relative to an explanation scheme. We employ this framework to consider various explanation schemes and argue that the preferred scheme depends on how much the user knows about the domain and the probability measure over the feature space.