Domain Adaptive Skin Lesion Classification via Conformal Ensemble of Vision Transformers

TL;DR

This paper introduces CE-ViTs, an ensemble of vision transformers, to improve domain adaptation and uncertainty estimation in skin lesion classification, demonstrating enhanced coverage and robustness across diverse datasets.

Contribution

The paper proposes a novel conformal ensemble framework using vision transformers trained on multiple datasets to improve domain adaptation and uncertainty quantification in skin lesion classification.

Findings

Achieves 90.38% coverage rate, a 9.95% improvement over single models.

Ensemble learning increases average prediction set size for challenging samples from 1.86 to 3.075.

Enhances robustness and reliability of skin lesion classification across datasets.

Abstract

Exploring the trustworthiness of deep learning models is crucial, especially in critical domains such as medical imaging decision support systems. Conformal prediction has emerged as a rigorous means of providing deep learning models with reliable uncertainty estimates and safety guarantees. However, conformal prediction results face challenges due to the backbone model's struggles in domain-shifted scenarios, such as variations in different sources. To aim this challenge, this paper proposes a novel framework termed Conformal Ensemble of Vision Transformers (CE-ViTs) designed to enhance image classification performance by prioritizing domain adaptation and model robustness, while accounting for uncertainty. The proposed method leverages an ensemble of vision transformer models in the backbone, trained on diverse datasets including HAM10000, Dermofit, and Skin Cancer ISIC datasets. This ensemble learning approach, calibrated through the combined mentioned datasets, aims to enhance domain adaptation through conformal learning. Experimental results underscore that the framework achieves a high coverage rate of 90.38\%, representing an improvement of 9.95\% compared to the HAM10000 model. This indicates a strong likelihood that the prediction set includes the true label compared to singular models. Ensemble learning in CE-ViTs significantly improves conformal prediction performance, increasing the average prediction set size for challenging misclassified samples from 1.86 to 3.075.