Malignancy Prediction and Lesion Identification from Clinical Dermatological Images

TL;DR

This paper introduces a two-stage machine learning framework for detecting and classifying skin lesions in dermatological images, supporting wide-field and focal images, and compares its performance to dermatologists.

Contribution

The study presents a novel two-stage approach that identifies lesions and predicts malignancy likelihood, outperforming alternative models and matching dermatologist performance.

Findings

Proposed model outperforms alternative architectures.

Image-based model surpasses clinical-only model.

Framework achieves dermatologist-level accuracy.

Abstract

We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images, which can be indistinctly acquired via smartphone or dermoscopy capture. Additionally, we do not assume that images contain single lesions, thus the framework supports both focal or wide-field images. Specifically, we propose a two-stage approach in which we first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy that can be used for high-level screening processes. Further, we consider augmenting the proposed approach with clinical covariates (from electronic health records) and publicly available data (the ISIC dataset). Comprehensive experiments validated on an independent test dataset demonstrate that i) the proposed approach outperforms alternative model architectures; ii) the model based on images outperforms a pure clinical model by a large margin, and the combination of images and clinical data does not significantly improves over the image-only model; and iii) the proposed framework offers comparable performance in terms of malignancy classification relative to three board certified dermatologists with different levels of experience.