A Quad-Step Approach to Uncertainty-Aware Deep Learning for Skin Cancer Classification

TL;DR

This paper evaluates deep learning models with transfer learning and uncertainty quantification for skin cancer classification, proposing a feature-fusion approach with a novel loss function to improve reliability and accuracy.

Contribution

It introduces a comprehensive benchmarking of pre-trained models with uncertainty quantification techniques and proposes a feature-fusion model trained with a predictive entropy loss for enhanced trustworthiness.

Findings

Ensemble methods with PCA-256 outperform other configurations.

Feature fusion of top extractors improves accuracy and reliability.

The proposed PE loss-based model surpasses prior methods in uncertainty-aware metrics.

Abstract

Accurate skin cancer diagnosis is vital for early treatment and improved patient outcomes. Deep learning (DL) models have shown promise in automating skin cancer classification, yet challenges remain due to data scarcity and limited uncertainty awareness. This study presents a comprehensive evaluation of DL-based skin lesion classification with transfer learning and uncertainty quantification (UQ) on the HAM10000 dataset. We benchmark several pre-trained feature extractors -- including CLIP variants, ResNet50, DenseNet121, VGG16, and EfficientNet-V2-Large -- combined with traditional classifiers such as SVM, XGBoost, and logistic regression. Multiple principal component analysis (PCA) settings (64, 128, 256, 512) are explored, with LAION CLIP ViT-H/14 and ViT-L/14 at PCA-256 achieving the strongest baseline results. In the UQ phase, Monte Carlo Dropout (MCD), Ensemble, and Ensemble Monte Carlo Dropout (EMCD) are applied and evaluated using uncertainty-aware metrics (UAcc, USen, USpe, UPre). Ensemble methods with PCA-256 provide the best balance between accuracy and reliability. Further improvements are obtained through feature fusion of top-performing extractors at PCA-256. Finally, we propose a feature-fusion based model trained with a predictive entropy (PE) loss function, which outperforms all prior configurations across both standard and uncertainty-aware evaluations, advancing trustworthy DL-based skin cancer diagnosis.