Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis

TL;DR

This paper proposes using controllably generated synthetic skin lesion images via GenAI to evaluate and improve fairness in skin cancer classifiers, addressing demographic biases in datasets.

Contribution

It introduces a method to generate realistic synthetic images for fairness testing of skin lesion classifiers, enabling bias detection across demographic groups.

Findings

Synthetic images show similar classification patterns to real images across models.

Realistic synthetic data can effectively evaluate model fairness.

The approach helps identify demographic biases in classifiers.

Abstract

Recent advances in deep learning and on-device inference could transform routine screening for skin cancers. Along with the anticipated benefits of this technology, potential dangers arise from unforeseen and inherent biases. A significant obstacle is building evaluation datasets that accurately reflect key demographics, including sex, age, and race, as well as other underrepresented groups. To address this, we train a state-of-the-art generative model to generate synthetic data in a controllable manner to assess the fairness of publicly available skin cancer classifiers. To evaluate whether synthetic images can be used as a fairness testing dataset, we prepare a real-image dataset (MILK10K) as a benchmark and compare the True Positive Rate result of three models (DeepGuide, MelaNet, and SkinLesionDensnet). As a result, the classification tendencies observed in each model when tested on real and generated images showed similar patterns across different attribute data sets. We confirm that highly realistic synthetic images facilitate model fairness verification.