One Size Fits None: Rethinking Fairness in Medical AI

TL;DR

This paper emphasizes the importance of subgroup-level evaluation in medical AI to identify performance disparities across patient groups, promoting fairer and more transparent clinical decision support systems.

Contribution

It advocates for subgroup-sensitive analysis of medical ML models to improve fairness and inform responsible deployment in healthcare.

Findings

Model performance varies significantly across patient subgroups.

Subgroup analysis reveals disparities hidden in overall metrics.

Performance insights can guide fairer clinical AI deployment.

Abstract

Machine learning (ML) models are increasingly used to support clinical decision-making. However, real-world medical datasets are often noisy, incomplete, and imbalanced, leading to performance disparities across patient subgroups. These differences raise fairness concerns, particularly when they reinforce existing disadvantages for marginalized groups. In this work, we analyze several medical prediction tasks and demonstrate how model performance varies with patient characteristics. While ML models may demonstrate good overall performance, we argue that subgroup-level evaluation is essential before integrating them into clinical workflows. By conducting a performance analysis at the subgroup level, differences can be clearly identified-allowing, on the one hand, for performance disparities to be considered in clinical practice, and on the other hand, for these insights to inform the responsible development of more effective models. Thereby, our work contributes to a practical discussion around the subgroup-sensitive development and deployment of medical ML models and the interconnectedness of fairness and transparency.