# Advancing skin cancer detection through deep learning and fusion of patient metadata and skin lesion images

## TL;DR

This paper introduces an AI framework that combines patient metadata and skin lesion images to improve skin cancer detection and reduce waiting times for diagnosis.

## Contribution

The novel contribution is an AI framework that fuses patient metadata with skin lesion images to enhance skin cancer classification accuracy and explainability.

## Key findings

- A fused AI model using metadata and images achieved 99.66% sensitivity and 74.45% specificity in skin lesion classification.
- Majority voting of AI models improved sensitivity to 99.50% and specificity to 82.72%, outperforming image-only methods.
- A soft-attention module was added to provide explainability for AI decisions, aiding healthcare professionals.

## Abstract

There has been a significant rise in skin cancer incidence during the last three decades and the waiting time for skin lesion assessment in both the NHS and private sectors in the UK has increased significantly. Therefore, to reduce waiting time and to make a faster decision, there is a need to develop automated methods that can be used to classify whether a skin lesion is suspicious or non-suspicious during teledermatology triage. In this study, we propose an AI framework that uses patient metadata together with image data to classify skin lesions into suspicious or non-suspicious categories. To evaluate our proposed approach, we collected 79,246 skin lesion images along with their 22 meta-features such as lesion size, lesion colour, lesion shape, patient age, and gender from 19,295 patients who attended a network of private skin cancer diagnostic centres across the UK. We developed three separate models for skin lesion classification: (1) an AI model using only metadata that achieved 85.24 ± 2.20% sensitivity and 61.12 ± 0.90% specificity; (2) an AI model using only images that achieved 99.72 ± 1.35% sensitivity and 63.22 ± 3.11% specificity; and (3) a fused model based on both metadata and images that achieved 99.66 ± 0.28% sensitivity and 74.45 ± 0.80% specificity. The decisions of the developed AI models were then fused through a majority voting technique, which achieved a sensitivity of 99.50 ± 1.18% and a specificity of 82.72 ± 1.64%, significantly outperforming the state-of-the-art methods that rely solely on image data. Furthermore, we add a post-processing step to explain AI model decisions by implementing a soft-attention module that provides essential explainability and supports healthcare professionals in informed decision-making. The developed AI framework has great potential for the detection of suspicious skin lesions. With a reduction in patient referrals for possible biopsies, waiting times for skin cancer diagnosis and treatment will be shortened, resulting in improved outcomes.