Assessing the Generalizability of Deep Neural Networks-Based Models for Black Skin Lesions

TL;DR

This study evaluates the generalizability of deep neural network models for diagnosing skin lesions in black skin, revealing poor performance due to dataset biases and emphasizing the need for diverse, inclusive datasets.

Contribution

The paper curates a dataset of acral skin lesions in black individuals and assesses model performance across skin tones, highlighting the importance of inclusivity in medical AI.

Findings

Models perform poorly on black skin lesions compared to white skin.

Current datasets lack diversity, leading to biased model outcomes.

Inclusive datasets are crucial for equitable AI-based diagnosis.

Abstract

Melanoma is the most severe type of skin cancer due to its ability to cause metastasis. It is more common in black people, often affecting acral regions: palms, soles, and nails. Deep neural networks have shown tremendous potential for improving clinical care and skin cancer diagnosis. Nevertheless, prevailing studies predominantly rely on datasets of white skin tones, neglecting to report diagnostic outcomes for diverse patient skin tones. In this work, we evaluate supervised and self-supervised models in skin lesion images extracted from acral regions commonly observed in black individuals. Also, we carefully curate a dataset containing skin lesions in acral regions and assess the datasets concerning the Fitzpatrick scale to verify performance on black skin. Our results expose the poor generalizability of these models, revealing their favorable performance for lesions on white skin. Neglecting to create diverse datasets, which necessitates the development of specialized models, is unacceptable. Deep neural networks have great potential to improve diagnosis, particularly for populations with limited access to dermatology. However, including black skin lesions is necessary to ensure these populations can access the benefits of inclusive technology.