Unsupervised Contrastive Learning based Transformer for Lung Nodule Detection

TL;DR

This paper introduces a self-supervised 3D transformer model utilizing contrastive learning for lung nodule detection in CT scans, aiming to improve accuracy and reduce false positives in computer-aided diagnosis systems.

Contribution

It presents a novel unsupervised contrastive pre-training approach for a 3D vision transformer tailored for lung nodule detection, enhancing performance over traditional CNNs.

Findings

Significant improvement in detection accuracy compared to 3D CNNs.

Effective pre-training with public CT images boosts model performance.

Reduces false positives in lung nodule screening.

Abstract

Early detection of lung nodules with computed tomography (CT) is critical for the longer survival of lung cancer patients and better quality of life. Computer-aided detection/diagnosis (CAD) is proven valuable as a second or concurrent reader in this context. However, accurate detection of lung nodules remains a challenge for such CAD systems and even radiologists due to not only the variability in size, location, and appearance of lung nodules but also the complexity of lung structures. This leads to a high false-positive rate with CAD, compromising its clinical efficacy. Motivated by recent computer vision techniques, here we present a self-supervised region-based 3D transformer model to identify lung nodules among a set of candidate regions. Specifically, a 3D vision transformer (ViT) is developed that divides a CT image volume into a sequence of non-overlap cubes, extracts embedding features from each cube with an embedding layer, and analyzes all embedding features with a self-attention mechanism for the prediction. To effectively train the transformer model on a relatively small dataset, the region-based contrastive learning method is used to boost the performance by pre-training the 3D transformer with public CT images. Our experiments show that the proposed method can significantly improve the performance of lung nodule screening in comparison with the commonly used 3D convolutional neural networks.