On the Role of Calibration in Benchmarking Algorithmic Fairness for Skin Cancer Detection

TL;DR

This paper emphasizes the importance of calibration metrics in benchmarking skin cancer detection AI models, revealing that high accuracy models may still have calibration issues affecting fairness across demographic groups.

Contribution

It introduces calibration as a key metric for assessing fairness in AI skin cancer detection models, supplementing traditional AUROC-based evaluations.

Findings

Models improve discriminative accuracy but often over-diagnose risk.

Calibration issues are prevalent when models are applied to new datasets.

Comprehensive auditing and metadata collection are essential for fairness.

Abstract

Artificial Intelligence (AI) models have demonstrated expert-level performance in melanoma detection, yet their clinical adoption is hindered by performance disparities across demographic subgroups such as gender, race, and age. Previous efforts to benchmark the performance of AI models have primarily focused on assessing model performance using group fairness metrics that rely on the Area Under the Receiver Operating Characteristic curve (AUROC), which does not provide insights into a model's ability to provide accurate estimates. In line with clinical assessments, this paper addresses this gap by incorporating calibration as a complementary benchmarking metric to AUROC-based fairness metrics. Calibration evaluates the alignment between predicted probabilities and observed event rates, offering deeper insights into subgroup biases. We assess the performance of the leading skin cancer detection algorithm of the ISIC 2020 Challenge on the ISIC 2020 Challenge dataset and the PROVE-AI dataset, and compare it with the second and third place models, focusing on subgroups defined by sex, race (Fitzpatrick Skin Tone), and age. Our findings reveal that while existing models enhance discriminative accuracy, they often over-diagnose risk and exhibit calibration issues when applied to new datasets. This study underscores the necessity for comprehensive model auditing strategies and extensive metadata collection to achieve equitable AI-driven healthcare solutions. All code is publicly available at https://github.com/bdominique/testing_strong_calibration.