Improving Fairness of Large Language Model-Based ICU Mortality Prediction via Case-Based Prompting

TL;DR

This paper introduces a prompt-based method to enhance fairness and accuracy in ICU mortality predictions using large language models, without retraining, by leveraging case-based guidance and debiasing strategies.

Contribution

The study presents CAse Prompting (CAP), a training-free framework that improves fairness and predictive performance in LLMs for clinical tasks through case-based prompts.

Findings

AUROC improved from 0.806 to 0.873

Prediction disparities reduced by over 90% across groups

Attention patterns remained highly consistent across demographic groups

Abstract

Accurately predicting mortality risk in intensive care unit (ICU) patients is essential for clinical decision-making. Although large language models (LLMs) show strong potential in structured medical prediction tasks, their outputs may exhibit biases related to demographic attributes such as sex, age, and race, limiting their reliability in fairness-critical clinical settings. Existing debiasing methods often degrade predictive performance, making it difficult to balance fairness and accuracy. In this study, we systematically analyze fairness issues in LLM-based ICU mortality prediction and propose a clinically adaptive prompting framework that improves both performance and fairness without model retraining. We first design a multi-dimensional bias assessment scheme to identify subgroup disparities. Based on this, we introduce CAse Prompting (CAP), a training-free framework that integrates existing debiasing strategies and further guides models using similar historical misprediction cases paired with correct outcomes to correct biased reasoning. We evaluate CAP on the MIMIC-IV dataset. Results show that AUROC improves from 0.806 to 0.873 and AUPRC from 0.497 to 0.694. Meanwhile, prediction disparities are substantially reduced across demographic groups, with reductions exceeding 90% in sex and certain White-Black comparisons. Feature reliance analysis further reveals highly consistent attention patterns across groups, with similarity above 0.98. These findings demonstrate that fairness and performance in LLM-based clinical prediction can be jointly optimized through carefully designed prompting, offering a practical paradigm for developing reliable and equitable clinical decision-support systems.