BronchoGAN: Anatomically consistent and domain-agnostic image-to-image translation for video bronchoscopy

TL;DR

BronchoGAN introduces an anatomically constrained, domain-agnostic image translation method for bronchoscopy images, leveraging foundation model-generated depth images to produce realistic, diverse, and robust synthetic airway images for clinical training and research.

Contribution

It presents a novel conditional GAN with anatomical constraints and depth image intermediates, enabling robust cross-domain bronchoscopy image translation with less reliance on training data.

Findings

Successful translation of images from various domains to realistic airway images

Improved FID, SSIM, and dice scores demonstrating enhanced image quality and anatomical preservation

Up to 0.43 increase in dice coefficient for synthetic images

Abstract

The limited availability of bronchoscopy images makes image synthesis particularly interesting for training deep learning models. Robust image translation across different domains -- virtual bronchoscopy, phantom as well as in-vivo and ex-vivo image data -- is pivotal for clinical applications. This paper proposes BronchoGAN introducing anatomical constraints for image-to-image translation being integrated into a conditional GAN. In particular, we force bronchial orifices to match across input and output images. We further propose to use foundation model-generated depth images as intermediate representation ensuring robustness across a variety of input domains establishing models with substantially less reliance on individual training datasets. Moreover our intermediate depth image representation allows to easily construct paired image data for training. Our experiments showed that input images from different domains (e.g. virtual bronchoscopy, phantoms) can be successfully translated to images mimicking realistic human airway appearance. We demonstrated that anatomical settings (i.e. bronchial orifices) can be robustly preserved with our approach which is shown qualitatively and quantitatively by means of improved FID, SSIM and dice coefficients scores. Our anatomical constraints enabled an improvement in the Dice coefficient of up to 0.43 for synthetic images. Through foundation models for intermediate depth representations, bronchial orifice segmentation integrated as anatomical constraints into conditional GANs we are able to robustly translate images from different bronchoscopy input domains. BronchoGAN allows to incorporate public CT scan data (virtual bronchoscopy) in order to generate large-scale bronchoscopy image datasets with realistic appearance. BronchoGAN enables to bridge the gap of missing public bronchoscopy images.