Uncovering Latent Bias in LLM-Based Emergency Department Triage Through Proxy Variables

TL;DR

This study uncovers hidden biases in large language models used for emergency department triage, revealing that they can discriminate based on proxy variables and alter perceived patient severity, highlighting the need for safer AI deployment.

Contribution

The paper introduces a novel analysis of bias in LLM-based medical triage systems using proxy variables and evaluates their discriminatory effects across multiple datasets.

Findings

LLMs exhibit bias mediated through proxy variables in ED triage.

Models tend to modify patient severity perceptions based on input tokens.

AI systems are influenced by noisy signals that do not reflect true patient acuity.

Abstract

Recent advances in large language models (LLMs) have enabled their integration into clinical decision-making; however, hidden biases against patients across racial, social, economic, and clinical backgrounds persist. In this study, we investigate bias in LLM-based medical AI systems applied to emergency department (ED) triage. We employ 32 patient-level proxy variables, each represented by paired positive and negative qualifiers, and evaluate their effects using both public (MIMIC-IV-ED Demo, MIMIC-IV Demo) and restricted-access credentialed (MIMIC-IV-ED and MIMIC-IV) datasets as appropriate~\cite{mimiciv_ed_demo,mimiciv_ed,mimiciv}. Our results reveal discriminatory behavior mediated through proxy variables in ED triage scenarios, as well as a systematic tendency for LLMs to modify perceived patient severity when specific tokens appear in the input context, regardless of whether they are framed positively or negatively. These findings indicate that AI systems is still imperfectly trained on noisy, sometimes non-causal signals that do not reliably reflect true patient acuity. Consequently, more needs to be done to ensure the safe and responsible deployment of AI technologies in clinical settings.