Mitigating the Risk of Health Inequity Exacerbated by Large Language Models

TL;DR

This paper highlights the risk that large language models may worsen health disparities by incorporating sensitive sociodemographic factors, and introduces EquityGuard, a framework to detect and reduce such inequities in medical AI applications.

Contribution

We propose EquityGuard, a novel framework to identify and mitigate health inequities caused by large language models in medical contexts.

Findings

EquityGuard effectively detects potential biases in LLM outputs.

Implementation of EquityGuard promotes equitable healthcare outcomes.

The framework reduces disparities related to sociodemographic factors.

Abstract

Recent advancements in large language models have demonstrated their potential in numerous medical applications, particularly in automating clinical trial matching for translational research and enhancing medical question answering for clinical decision support. However, our study shows that incorporating non decisive sociodemographic factors such as race, sex, income level, LGBT+ status, homelessness, illiteracy, disability, and unemployment into the input of LLMs can lead to incorrect and harmful outputs for these populations. These discrepancies risk exacerbating existing health disparities if LLMs are widely adopted in healthcare. To address this issue, we introduce EquityGuard, a novel framework designed to detect and mitigate the risk of health inequities in LLM based medical applications. Our evaluation demonstrates its efficacy in promoting equitable outcomes across diverse populations.