Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings

TL;DR

This paper investigates biases in clinical contextual word embeddings, revealing significant disparities across demographic groups and evaluating methods to mitigate these biases in medical AI applications.

Contribution

It introduces methods to quantify biases in clinical embeddings, assesses their impact on downstream tasks, and discusses limitations of debiasing techniques in healthcare contexts.

Findings

Significant performance disparities across demographic groups.

Identification of dangerous latent relationships in embeddings.

Limitations of adversarial debiasing in clinical models.

Abstract

In this work, we examine the extent to which embeddings may encode marginalized populations differently, and how this may lead to a perpetuation of biases and worsened performance on clinical tasks. We pretrain deep embedding models (BERT) on medical notes from the MIMIC-III hospital dataset, and quantify potential disparities using two approaches. First, we identify dangerous latent relationships that are captured by the contextual word embeddings using a fill-in-the-blank method with text from real clinical notes and a log probability bias score quantification. Second, we evaluate performance gaps across different definitions of fairness on over 50 downstream clinical prediction tasks that include detection of acute and chronic conditions. We find that classifiers trained from BERT representations exhibit statistically significant differences in performance, often favoring the majority group with regards to gender, language, ethnicity, and insurance status. Finally, we explore shortcomings of using adversarial debiasing to obfuscate subgroup information in contextual word embeddings, and recommend best practices for such deep embedding models in clinical settings.