From Real Artifacts to Virtual Reference: A Robust Framework for Translating Endoscopic Images

TL;DR

This paper introduces a robust image translation framework for endoscopic images that effectively handles artifacts and noise, improving domain adaptation between noisy intra-operative videos and clean virtual pre-operative images for surgical guidance.

Contribution

It proposes a novel local-global translation framework and a noise-resilient feature extraction strategy, advancing artifact-robust domain adaptation in medical image analysis.

Findings

Significantly outperforms existing methods on clinical datasets.

Demonstrates improved pose estimation accuracy during surgery.

Provides a new benchmark for artifact-resilient endoscopic image translation.

Abstract

Domain adaptation, which bridges the distributions across different modalities, plays a crucial role in multimodal medical image analysis. In endoscopic imaging, combining pre-operative data with intra-operative imaging is important for surgical planning and navigation. However, existing domain adaptation methods are hampered by distribution shift caused by in vivo artifacts, necessitating robust techniques for aligning noisy and artifact abundant patient endoscopic videos with clean virtual images reconstructed from pre-operative tomographic data for pose estimation during intraoperative guidance. This paper presents an artifact-resilient image translation method and an associated benchmark for this purpose. The method incorporates a novel ``local-global'' translation framework and a noise-resilient feature extraction strategy. For the former, it decouples the image translation process into a local step for feature denoising, and a global step for global style transfer. For feature extraction, a new contrastive learning strategy is proposed, which can extract noise-resilient features for establishing robust correspondence across domains. Detailed validation on both public and in-house clinical datasets has been conducted, demonstrating significantly improved performance compared to the current state-of-the-art.