From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions

TL;DR

This paper introduces DCEM, an EM-based algorithm designed to address biased selective labels in clinical decision-making, effectively reducing bias while maintaining model performance.

Contribution

The paper proposes a novel EM framework, DCEM, inspired by causal models, to mitigate bias from disparate censorship in selective labeling scenarios.

Findings

DCEM reduces bias in synthetic data experiments.

DCEM maintains discriminative performance while mitigating bias.

Effective in clinical sepsis classification data.

Abstract

Selective labels occur when label observations are subject to a decision-making process; e.g., diagnoses that depend on the administration of laboratory tests. We study a clinically-inspired selective label problem called disparate censorship, where labeling biases vary across subgroups and unlabeled individuals are imputed as "negative" (i.e., no diagnostic test = no illness). Machine learning models naively trained on such labels could amplify labeling bias. Inspired by causal models of selective labels, we propose Disparate Censorship Expectation-Maximization (DCEM), an algorithm for learning in the presence of disparate censorship. We theoretically analyze how DCEM mitigates the effects of disparate censorship on model performance. We validate DCEM on synthetic data, showing that it improves bias mitigation (area between ROC curves) without sacrificing discriminative performance (AUC) compared to baselines. We achieve similar results in a sepsis classification task using clinical data.