Algorithmic Bias, Generalist Models,and Clinical Medicine

TL;DR

This paper explores how the shift from narrow to generalist models in clinical machine learning introduces new types of algorithmic bias, affecting interpretability and requiring updated mitigation strategies.

Contribution

It analyzes differences in biases between traditional clinical models and emerging generalist models, offering practical recommendations for bias mitigation.

Findings

Generalist models show performance improvements over traditional models.

New biases emerge in generalist models due to their broad training data.

Bias mitigation strategies need adaptation for generalist models.

Abstract

The technical landscape of clinical machine learning is shifting in ways that destabilize pervasive assumptions about the nature and causes of algorithmic bias. On one hand, the dominant paradigm in clinical machine learning is narrow in the sense that models are trained on biomedical datasets for particular clinical tasks such as diagnosis and treatment recommendation. On the other hand, the emerging paradigm is generalist in the sense that general-purpose language models such as Google's BERT and PaLM are increasingly being adapted for clinical use cases via prompting or fine-tuning on biomedical datasets. Many of these next-generation models provide substantial performance gains over prior clinical models, but at the same time introduce novel kinds of algorithmic bias and complicate the explanatory relationship between algorithmic biases and biases in training data. This paper articulates how and in what respects biases in generalist models differ from biases in prior clinical models, and draws out practical recommendations for algorithmic bias mitigation.