EndoUDA: A modality independent segmentation approach for endoscopy imaging

TL;DR

EndoUDA introduces an unsupervised domain adaptation method for endoscopy image segmentation that generalizes across different imaging modalities without requiring labeled data for each modality.

Contribution

This work presents a novel UDA-based segmentation approach combining VAE, U-Net, and EfficientNet-B4 that effectively generalizes to unseen endoscopy modalities.

Findings

Successfully generalizes to unseen NBI modality from WLI training data

Outperforms naive supervised and existing UDA segmentation methods

Effective on both upper and lower GI endoscopy datasets

Abstract

Gastrointestinal (GI) cancer precursors require frequent monitoring for risk stratification of patients. Automated segmentation methods can help to assess risk areas more accurately, and assist in therapeutic procedures or even removal. In clinical practice, addition to the conventional white-light imaging (WLI), complimentary modalities such as narrow-band imaging (NBI) and fluorescence imaging are used. While, today most segmentation approaches are supervised and only concentrated on a single modality dataset, this work exploits to use a target-independent unsupervised domain adaptation (UDA) technique that is capable to generalize to an unseen target modality. In this context, we propose a novel UDA-based segmentation method that couples the variational autoencoder and U-Net with a common EfficientNet-B4 backbone, and uses a joint loss for latent-space optimization for target samples. We show that our model can generalize to unseen target NBI (target) modality when trained using only WLI (source) modality. Our experiments on both upper and lower GI endoscopy data show the effectiveness of our approach compared to naive supervised approach and state-of-the-art UDA segmentation methods.