Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation

TL;DR

This paper introduces a novel Structure-to-Image approach for zero-shot depth estimation in colonoscopy, effectively bridging the domain gap between simulated and real images by leveraging phase congruency and cross-level structure constraints.

Contribution

It presents the first use of phase congruency in colonoscopic domain adaptation and develops a cross-level structure constraint for improved depth estimation accuracy.

Findings

Achieved up to 44.18% RMSE reduction in zero-shot evaluations.

Outperformed existing image translation methods in colonoscopy depth estimation.

Demonstrated effectiveness of the proposed method on a public phantom dataset.

Abstract

Monocular depth estimation (MDE) for colonoscopy is hampered by the domain gap between simulated and real-world images. Existing image-to-image translation methods, which use depth as a posterior constraint, often produce structural distortions and specular highlights by failing to balance realism with structure consistency. To address this, we propose a Structure-to-Image paradigm that transforms the depth map from a passive constraint into an active generative foundation. We are the first to introduce phase congruency to colonoscopic domain adaptation and design a cross-level structure constraint to co-optimize geometric structures and fine-grained details like vascular textures. In zero-shot evaluations conducted on a publicly available phantom dataset, the MDE model that was fine-tuned on our generated data achieved a maximum reduction of 44.18% in RMSE compared to competing methods. Our code is available at https://github.com/YyangJJuan/PC-S2I.git.