Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation

TL;DR

This paper introduces a data-centric method to detect and mitigate label bias in image segmentation datasets without needing unbiased ground truth, improving fairness across demographic groups.

Contribution

It adapts Confident Learning for segmentation, enabling bias detection directly from training data and leveraging feature space analysis for bias mitigation.

Findings

The method detects label bias without clean labels.

Bias influences subgroup separability in feature space.

Mitigation improves equitable performance across groups.

Abstract

Labeled datasets reflect the biases of their annotation pipelines, which sometimes introduce label bias: group-conditional label errors that cause systematic performance disparities across demographic subgroups. Label bias in image segmentation remains underexplored, as even detecting it typically requires clean, unbiased annotations, which are not readily available. We present a data-centric adaptation of Confident Learning to segmentation, allowing detection of label bias directly in the training data without a clean, unbiased ground truth. By comparing the provided training labels to the model's confident predictions, we isolate directional errors that quantify the presence and nature of bias, where standard overlap metrics like Dice fail. We further show that label bias influences subgroup separability in the encoder's feature space, an artifact we leverage for bias mitigation rather than suppressing it. We evaluate three datasets, spanning from synthetic to real-life bias, showing how our framework reliably detects and mitigates bias without access to clean labels, achieving equitable performance across experimental conditions.