ChatGPT Exhibits Gender and Racial Biases in Acute Coronary Syndrome Management

TL;DR

This study investigates gender and racial biases in ChatGPT 3.5's clinical management of acute coronary syndrome, revealing disparities in treatment recommendations and showing that prompting for explanations can reduce biases.

Contribution

It is among the first to demonstrate that LLMs exhibit biases affecting clinical decisions and that prompting strategies can mitigate these biases.

Findings

Biases lead to reduced guideline-based management for female and minority patients.

Prompting ChatGPT to explain its reasoning improves accuracy and reduces biases.

Disparities are most pronounced during unstable angina management.

Abstract

Recent breakthroughs in large language models (LLMs) have led to their rapid dissemination and widespread use. One early application has been to medicine, where LLMs have been investigated to streamline clinical workflows and facilitate clinical analysis and decision-making. However, a leading barrier to the deployment of Artificial Intelligence (AI) and in particular LLMs has been concern for embedded gender and racial biases. Here, we evaluate whether a leading LLM, ChatGPT 3.5, exhibits gender and racial bias in clinical management of acute coronary syndrome (ACS). We find that specifying patients as female, African American, or Hispanic resulted in a decrease in guideline recommended medical management, diagnosis, and symptom management of ACS. Most notably, the largest disparities were seen in the recommendation of coronary angiography or stress testing for the diagnosis and further intervention of ACS and recommendation of high intensity statins. These disparities correlate with biases that have been observed clinically and have been implicated in the differential gender and racial morbidity and mortality outcomes of ACS and coronary artery disease. Furthermore, we find that the largest disparities are seen during unstable angina, where fewer explicit clinical guidelines exist. Finally, we find that through asking ChatGPT 3.5 to explain its reasoning prior to providing an answer, we are able to improve clinical accuracy and mitigate instances of gender and racial biases. This is among the first studies to demonstrate that the gender and racial biases that LLMs exhibit do in fact affect clinical management. Additionally, we demonstrate that existing strategies that improve LLM performance not only improve LLM performance in clinical management, but can also be used to mitigate gender and racial biases.