Explaining k-Nearest Neighbors: Abductive and Counterfactual Explanations

TL;DR

This paper investigates the interpretability of k-Nearest Neighbors classifiers by analyzing abductive and counterfactual explanations from a feature perspective, providing complexity insights and practical solution methods.

Contribution

It offers a theoretical analysis of explanation complexity for k-NN classifiers and demonstrates practical computation approaches using Integer Quadratic Programming and SAT solving.

Findings

Complexity results vary between discrete and continuous features.

Counterfactual explanations can be computed using SAT and Integer Quadratic Programming.

Understanding feature influence enhances interpretability of k-NN models.

Abstract

Despite the wide use of $k$-Nearest Neighbors as classification models, their explainability properties remain poorly understood from a theoretical perspective. While nearest neighbors classifiers offer interpretability from a ``data perspective'', in which the classification of an input vector $\bar{x}$ is explained by identifying the vectors $\bar{v}_1, \ldots, \bar{v}_k$ in the training set that determine the classification of $\bar{x}$, we argue that such explanations can be impractical in high-dimensional applications, where each vector has hundreds or thousands of features and it is not clear what their relative importance is. Hence, we focus on understanding nearest neighbor classifications through a ``feature perspective'', in which the goal is to identify how the values of the features in $\bar{x}$ affect its classification. Concretely, we study abductive explanations such as ``minimum sufficient reasons'', which correspond to sets of features in $\bar{x}$ that are enough to guarantee its classification, and counterfactual explanations based on the minimum distance feature changes one would have to perform in $\bar{x}$ to change its classification. We present a detailed landscape of positive and negative complexity results for counterfactual and abductive explanations, distinguishing between discrete and continuous feature spaces, and considering the impact of the choice of distance function involved. Finally, we show that despite some negative complexity results, Integer Quadratic Programming and SAT solving allow for computing explanations in practice.