Self-Supervised and Semi-Supervised Polyp Segmentation using Synthetic Data

TL;DR

This paper introduces a novel approach combining synthetic data, image translation, and pseudo-labeling to improve polyp segmentation, achieving state-of-the-art results with less reliance on manual annotations.

Contribution

It proposes Pl-CUT-Seg and Pl-CUT-Seg+ models that leverage synthetic data and unlabeled samples for effective semi-supervised polyp segmentation.

Findings

Achieves state-of-the-art results on benchmark datasets.

Effectively reduces the need for manual annotations.

Demonstrates the benefit of synthetic data and domain adaptation.

Abstract

Early detection of colorectal polyps is of utmost importance for their treatment and for colorectal cancer prevention. Computer vision techniques have the potential to aid professionals in the diagnosis stage, where colonoscopies are manually carried out to examine the entirety of the patient's colon. The main challenge in medical imaging is the lack of data, and a further challenge specific to polyp segmentation approaches is the difficulty of manually labeling the available data: the annotation process for segmentation tasks is very time-consuming. While most recent approaches address the data availability challenge with sophisticated techniques to better exploit the available labeled data, few of them explore the self-supervised or semi-supervised paradigm, where the amount of labeling required is greatly reduced. To address both challenges, we leverage synthetic data and propose an end-to-end model for polyp segmentation that integrates real and synthetic data to artificially increase the size of the datasets and aid the training when unlabeled samples are available. Concretely, our model, Pl-CUT-Seg, transforms synthetic images with an image-to-image translation module and combines the resulting images with real images to train a segmentation model, where we use model predictions as pseudo-labels to better leverage unlabeled samples. Additionally, we propose PL-CUT-Seg+, an improved version of the model that incorporates targeted regularization to address the domain gap between real and synthetic images. The models are evaluated on standard benchmarks for polyp segmentation and reach state-of-the-art results in the self- and semi-supervised setups.