CLTS-GAN: Color-Lighting-Texture-Specular Reflection Augmentation for Colonoscopy

TL;DR

CLTS-GAN is a deep learning model that synthesizes colonoscopy-specific variations in color, lighting, texture, and reflections to enhance training data and improve polyp detection and segmentation.

Contribution

We introduce CLTS-GAN, a novel GAN-based method for controllable augmentation of colonoscopy video frames, addressing limitations of previous preprocessing or annotation-based approaches.

Findings

Enhanced polyp detection and segmentation accuracy.

Improved training data diversity for colonoscopy analysis.

Facilitated development of OC simulators for training.

Abstract

Automated analysis of optical colonoscopy (OC) video frames (to assist endoscopists during OC) is challenging due to variations in color, lighting, texture, and specular reflections. Previous methods either remove some of these variations via preprocessing (making pipelines cumbersome) or add diverse training data with annotations (but expensive and time-consuming). We present CLTS-GAN, a new deep learning model that gives fine control over color, lighting, texture, and specular reflection synthesis for OC video frames. We show that adding these colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods as well as drive next generation of OC simulators for training medical students. The code and pre-trained models for CLTS-GAN are available on Computational Endoscopy Platform GitHub (https://github.com/nadeemlab/CEP).