Prediction Sensitivity: Continual Audit of Counterfactual Fairness in Deployed Classifiers

TL;DR

This paper introduces prediction sensitivity, a method for ongoing auditing of counterfactual fairness in deployed classifiers, capable of detecting individual fairness violations without needing protected attributes at prediction time.

Contribution

It proposes a novel approach for continual fairness auditing that evaluates counterfactual fairness post-deployment without requiring protected attribute data.

Findings

Prediction sensitivity effectively detects counterfactual fairness violations.

The method leverages feature correlations without protected attribute data.

Empirical results validate its utility in real-world classifiers.

Abstract

As AI-based systems increasingly impact many areas of our lives, auditing these systems for fairness is an increasingly high-stakes problem. Traditional group fairness metrics can miss discrimination against individuals and are difficult to apply after deployment. Counterfactual fairness describes an individualized notion of fairness but is even more challenging to evaluate after deployment. We present prediction sensitivity, an approach for continual audit of counterfactual fairness in deployed classifiers. Prediction sensitivity helps answer the question: would this prediction have been different, if this individual had belonged to a different demographic group -- for every prediction made by the deployed model. Prediction sensitivity can leverage correlations between protected status and other features and does not require protected status information at prediction time. Our empirical results demonstrate that prediction sensitivity is effective for detecting violations of counterfactual fairness.