FEDD -- Fair, Efficient, and Diverse Diffusion-based Lesion Segmentation and Malignancy Classification

TL;DR

FEDD is a diffusion-based framework that improves skin lesion segmentation and malignancy classification, especially in data-scarce and diverse skin tone scenarios, achieving state-of-the-art results with limited labeled data.

Contribution

Introduces FEDD, a novel diffusion-based method that enhances fairness, efficiency, and diversity in dermatology image analysis with minimal labeled data.

Findings

Improves segmentation IoU by up to 0.18 with limited labeled data.

Achieves 81% malignancy classification accuracy with only 10% labeled data.

Demonstrates high efficiency and fairness across skin tones and rare conditions.

Abstract

Skin diseases affect millions of people worldwide, across all ethnicities. Increasing diagnosis accessibility requires fair and accurate segmentation and classification of dermatology images. However, the scarcity of annotated medical images, especially for rare diseases and underrepresented skin tones, poses a challenge to the development of fair and accurate models. In this study, we introduce a Fair, Efficient, and Diverse Diffusion-based framework for skin lesion segmentation and malignancy classification. FEDD leverages semantically meaningful feature embeddings learned through a denoising diffusion probabilistic backbone and processes them via linear probes to achieve state-of-the-art performance on Diverse Dermatology Images (DDI). We achieve an improvement in intersection over union of 0.18, 0.13, 0.06, and 0.07 while using only 5%, 10%, 15%, and 20% labeled samples, respectively. Additionally, FEDD trained on 10% of DDI demonstrates malignancy classification accuracy of 81%, 14% higher compared to the state-of-the-art. We showcase high efficiency in data-constrained scenarios while providing fair performance for diverse skin tones and rare malignancy conditions. Our newly annotated DDI segmentation masks and training code can be found on https://github.com/hectorcarrion/fedd.