Diagnosing failures of fairness transfer across distribution shift in real-world medical settings

TL;DR

This paper introduces a causal framework using conditional independence tests to diagnose and understand fairness transfer failures under distribution shifts in healthcare, aiding in safer ML deployment.

Contribution

It proposes a novel causal approach to empirically assess distribution shift structures, improving diagnosis of fairness transfer failures in real-world medical settings.

Findings

Conditional independence tests help diagnose fairness transfer failures.

Knowledge of shift structure guides effective mitigation strategies.

Real-world shifts can be more complex than assumed in prior literature.

Abstract

Diagnosing and mitigating changes in model fairness under distribution shift is an important component of the safe deployment of machine learning in healthcare settings. Importantly, the success of any mitigation strategy strongly depends on the structure of the shift. Despite this, there has been little discussion of how to empirically assess the structure of a distribution shift that one is encountering in practice. In this work, we adopt a causal framing to motivate conditional independence tests as a key tool for characterizing distribution shifts. Using our approach in two medical applications, we show that this knowledge can help diagnose failures of fairness transfer, including cases where real-world shifts are more complex than is often assumed in the literature. Based on these results, we discuss potential remedies at each step of the machine learning pipeline.