Sufficient reasons for classifier decisions in the presence of constraints

TL;DR

This paper extends the theory of explaining binary classifier decisions by incorporating data constraints, resulting in more concise and accurate reasons for classifications, especially when certain feature combinations are invalid or unobservable.

Contribution

It introduces a generalization of prime-implicant based explanations that explicitly accounts for data constraints, improving explanation succinctness and relevance.

Findings

Constraints lead to more concise reasons

Reasons considering constraints subsume unconstrained reasons

Empirical validation on synthetic and real data

Abstract

Recent work has unveiled a theory for reasoning about the decisions made by binary classifiers: a classifier describes a Boolean function, and the reasons behind an instance being classified as positive are the prime-implicants of the function that are satisfied by the instance. One drawback of these works is that they do not explicitly treat scenarios where the underlying data is known to be constrained, e.g., certain combinations of features may not exist, may not be observable, or may be required to be disregarded. We propose a more general theory, also based on prime-implicants, tailored to taking constraints into account. The main idea is to view classifiers in the presence of constraints as describing partial Boolean functions, i.e., that are undefined on instances that do not satisfy the constraints. We prove that this simple idea results in reasons that are no less (and sometimes more) succinct. That is, not taking constraints into account (e.g., ignored, or taken as negative instances) results in reasons that are subsumed by reasons that do take constraints into account. We illustrate this improved parsimony on synthetic classifiers and classifiers learned from real data.