An Attention Based Pipeline for Identifying Pre-Cancer Lesions in Head and Neck Clinical Images

TL;DR

This paper introduces an attention-based pipeline utilizing vision transformers and multiple instance learning to detect, segment, and classify pre-cancerous lesions in head and neck clinical images, aiming for early diagnosis.

Contribution

It presents a novel combination of vision transformer-based segmentation and MIL-based classification for improved lesion detection and diagnosis in clinical images.

Findings

Segmentation model achieves up to 82% overlap accuracy on external data.

Classification model attains 85% F1-score on internal test set.

An app was developed for lesion segmentation using smart devices.

Abstract

Early detection of cancer can help improve patient prognosis by early intervention. Head and neck cancer is diagnosed in specialist centres after a surgical biopsy, however, there is a potential for these to be missed leading to delayed diagnosis. To overcome these challenges, we present an attention based pipeline that identifies suspected lesions, segments, and classifies them as non-dysplastic, dysplastic and cancerous lesions. We propose (a) a vision transformer based Mask R-CNN network for lesion detection and segmentation of clinical images, and (b) Multiple Instance Learning (MIL) based scheme for classification. Current results show that the segmentation model produces segmentation masks and bounding boxes with up to 82% overlap accuracy score on unseen external test data and surpassing reviewed segmentation benchmarks. Next, a classification F1-score of 85% on the internal cohort test set. An app has been developed to perform lesion segmentation taken via a smart device. Future work involves employing endoscopic video data for precise early detection and prognosis.