Achieving Reliable and Fair Skin Lesion Diagnosis via Unsupervised Domain Adaptation

TL;DR

This paper demonstrates that unsupervised domain adaptation using multiple sources can improve the reliability and fairness of skin lesion diagnosis systems, especially when labeled data are scarce.

Contribution

It introduces and evaluates three UDA schemes for skin lesion classification, showing effectiveness in enhancing accuracy and fairness without explicit fairness techniques.

Findings

UDA with multiple sources improves classification accuracy.

UDA reduces bias against minority groups.

Test error correlates with label shift in multi-class tasks.

Abstract

The development of reliable and fair diagnostic systems is often constrained by the scarcity of labeled data. To address this challenge, our work explores the feasibility of unsupervised domain adaptation (UDA) to integrate large external datasets for developing reliable classifiers. The adoption of UDA with multiple sources can simultaneously enrich the training set and bridge the domain gap between different skin lesion datasets, which vary due to distinct acquisition protocols. Particularly, UDA shows practical promise for improving diagnostic reliability when training with a custom skin lesion dataset, where only limited labeled data are available from the target domain. In this study, we investigate three UDA training schemes based on source data utilization: single-source, combined-source, and multi-source UDA. Our findings demonstrate the effectiveness of applying UDA on multiple sources for binary and multi-class classification. A strong correlation between test error and label shift in multi-class tasks has been observed in the experiment. Crucially, our study shows that UDA can effectively mitigate bias against minority groups and enhance fairness in diagnostic systems, while maintaining superior classification performance. This is achieved even without directly implementing fairness-focused techniques. This success is potentially attributed to the increased and well-adapted demographic information obtained from multiple sources.