Melanoma Diagnosis with Spatio-Temporal Feature Learning on Sequential Dermoscopic Images

TL;DR

This study introduces a spatio-temporal deep learning framework for melanoma diagnosis that leverages sequential dermoscopic images to improve accuracy by capturing lesion evolution over time.

Contribution

The paper presents a novel two-stream network architecture that simultaneously learns appearance and temporal changes from sequential dermoscopic images for melanoma detection.

Findings

Achieved an AUC of 74.34%, outperforming single-image methods.

Improved diagnostic accuracy by approximately 8% over single-image approaches.

Surpassed LSTM-based sequence models by about 6% in AUC.

Abstract

Existing studies for automated melanoma diagnosis are based on single-time point images of lesions. However, melanocytic lesions de facto are progressively evolving and, moreover, benign lesions can progress into malignant melanoma. Ignoring cross-time morphological changes of lesions thus may lead to misdiagnosis in borderline cases. Based on the fact that dermatologists diagnose ambiguous skin lesions by evaluating the dermoscopic changes over time via follow-up examination, in this study, we propose an automated framework for melanoma diagnosis using sequential dermoscopic images. To capture the spatio-temporal characterization of dermoscopic evolution, we construct our model in a two-stream network architecture which capable of simultaneously learning appearance representations of individual lesions while performing temporal reasoning on both raw pixels difference and abstract features difference. We collect 184 cases of serial dermoscopic image data, which consists of histologically confirmed 92 benign lesions and 92 melanoma lesions, to evaluate the effectiveness of the proposed method. Our model achieved AUC of 74.34%, which is ~8% higher than that of only using single images and ~6% higher than the widely used sequence learning model based on LSTM.