Colorimeter-Supervised Skin Tone Estimation from Dermatoscopic Images for Fairness Auditing

TL;DR

This paper introduces a neural network-based method for estimating skin tone from dermatoscopic images, validated against colorimeter measurements, to facilitate fairness auditing in clinical diagnosis models.

Contribution

It presents the first dermatoscopic skin-tone estimation neural network validated against colorimeter data, enabling reliable bias detection across skin tones.

Findings

High agreement with human annotations for Fitzpatrick skin types.

ITA predictions align closely with colorimeter measurements.

Less than 1% of subjects are Fitzpatrick types V and VI in studied datasets.

Abstract

Neural-network-based diagnosis from dermatoscopic images is increasingly used for clinical decision support, yet studies report performance disparities across skin tones. Fairness auditing of these models is limited by the lack of reliable skin-tone annotations in public dermatoscopy datasets. We address this gap with neural networks that predict Fitzpatrick skin type via ordinal regression and the Individual Typology Angle (ITA) via color regression, using in-person Fitzpatrick labels and colorimeter measurements as targets. We further leverage extensive pretraining on synthetic and real dermatoscopic and clinical images. The Fitzpatrick model achieves agreement comparable to human crowdsourced annotations, and ITA predictions show high concordance with colorimeter-derived ITA, substantially outperforming pixel-averaging approaches. Applying these estimators to ISIC 2020 and MILK10k, we find that fewer than 1% of subjects belong to Fitzpatrick types V and VI. We release code and pretrained models as an open-source tool for rapid skin-tone annotation and bias auditing. This is, to our knowledge, the first dermatoscopic skin-tone estimation neural network validated against colorimeter measurements, and it supports growing evidence of clinically relevant performance gaps across skin-tone groups.