An intersectional framework for counterfactual fairness in risk prediction

TL;DR

This paper introduces an intersectional counterfactual fairness framework for health risk prediction models, addressing multiple group fairness and clinical application challenges with new metrics and inference tools, demonstrated on COVID-19 data.

Contribution

It develops a novel intersectional fairness metric and estimation framework tailored for clinical risk prediction, overcoming limitations of existing fairness methods.

Findings

The framework effectively measures intersectional unfairness in health models.

Application to COVID-19 risk prediction reveals significant fairness issues.

Provides statistical inference tools for assessing model fairness.

Abstract

Along with the increasing availability of health data has come the rise of data-driven models to inform decision-making and policy. These models have the potential to benefit both patients and health care providers but can also exacerbate health inequities. Existing "algorithmic fairness" methods for measuring and correcting model bias fall short of what is needed for health policy in two key ways. First, methods typically focus on a single grouping along which discrimination may occur rather than considering multiple, intersecting groups. Second, in clinical applications, risk prediction is typically used to guide treatment, creating distinct statistical issues that invalidate most existing techniques. We present summary unfairness metrics that build on existing techniques in "counterfactual fairness" to address both challenges. We also develop a complete framework of estimation and inference tools for our metrics, including the unfairness value ("u-value"), used to determine the relative extremity of unfairness, and standard errors and confidence intervals employing an alternative to the standard bootstrap. We demonstrate application of our framework to a COVID-19 risk prediction model deployed in a major Midwestern health system.