Rethinking Distance Metrics for Counterfactual Explainability

TL;DR

This paper introduces a new distance metric for counterfactual explanations in machine learning, emphasizing joint sampling with the reference to better capture data distribution and dependencies among covariates.

Contribution

It proposes a novel framing for counterfactual generation that considers joint sampling, leading to a tailored distance metric for improved explainability.

Findings

The new metric captures nuanced dependencies among covariates.

Joint sampling framing improves counterfactual quality.

Quantitative and qualitative analyses validate the approach.

Abstract

Counterfactual explanations have been a popular method of post-hoc explainability for a variety of settings in Machine Learning. Such methods focus on explaining classifiers by generating new data points that are similar to a given reference, while receiving a more desirable prediction. In this work, we investigate a framing for counterfactual generation methods that considers counterfactuals not as independent draws from a region around the reference, but as jointly sampled with the reference from the underlying data distribution. Through this framing, we derive a distance metric, tailored for counterfactual similarity that can be applied to a broad range of settings. Through both quantitative and qualitative analyses of counterfactual generation methods, we show that this framing allows us to express more nuanced dependencies among the covariates.