Joint one-sided synthetic unpaired image translation and segmentation for colorectal cancer prevention

TL;DR

This paper introduces CUT-seg, a novel joint training method combining image translation and segmentation that produces realistic synthetic colon images and polyp segmentation with minimal real data, advancing medical image analysis.

Contribution

The paper presents a new joint training framework using one-sided translation models for efficient synthetic image generation and segmentation in colorectal cancer prevention.

Findings

Achieves high segmentation accuracy with only one real image.

Requires less computational resources than traditional methods.

Provides a large synthetic dataset for colon image analysis.

Abstract

Deep learning has shown excellent performance in analysing medical images. However, datasets are difficult to obtain due privacy issues, standardization problems, and lack of annotations. We address these problems by producing realistic synthetic images using a combination of 3D technologies and generative adversarial networks. We propose CUT-seg, a joint training where a segmentation model and a generative model are jointly trained to produce realistic images while learning to segment polyps. We take advantage of recent one-sided translation models because they use significantly less memory, allowing us to add a segmentation model in the training loop. CUT-seg performs better, is computationally less expensive, and requires less real images than other memory-intensive image translation approaches that require two stage training. Promising results are achieved on five real polyp segmentation datasets using only one real image and zero real annotations. As a part of this study we release Synth-Colon, an entirely synthetic dataset that includes 20000 realistic colon images and additional details about depth and 3D geometry: https://enric1994.github.io/synth-colon