Generalizability vs. Counterfactual Explainability Trade-Off

TL;DR

This paper explores the inherent trade-off between a model's ability to generalize and its counterfactual explainability, introducing a new measure called $ ext{ extepsilon}$-VCP to quantify this relationship.

Contribution

It introduces $ ext{ extepsilon}$-VCP, a theoretical framework linking model overfitting to increased counterfactual explainability, supported by empirical validation.

Findings

$ ext{ extepsilon}$-VCP increases with overfitting

Poor generalization correlates with easier counterfactual generation

$ ext{ extepsilon}$-VCP can serve as a proxy for overfitting

Abstract

In this work, we investigate the relationship between model generalization and counterfactual explainability in supervised learning. We introduce the notion of $\varepsilon$-valid counterfactual probability ($\varepsilon$-VCP) -- the probability of finding perturbations of a data point within its $\varepsilon$-neighborhood that result in a label change. We provide a theoretical analysis of $\varepsilon$-VCP in relation to the geometry of the model's decision boundary, showing that $\varepsilon$-VCP tends to increase with model overfitting. Our findings establish a rigorous connection between poor generalization and the ease of counterfactual generation, revealing an inherent trade-off between generalization and counterfactual explainability. Empirical results validate our theory, suggesting $\varepsilon$-VCP as a practical proxy for quantitatively characterizing overfitting.