Self-supervised Monocular Depth and Pose Estimation for Endoscopy with Generative Latent Priors

TL;DR

This paper introduces a self-supervised framework for monocular depth and pose estimation in endoscopy, utilizing generative latent priors and a VAE to improve accuracy and robustness in challenging GI tract conditions.

Contribution

It presents a novel approach combining a Generative Latent Bank and a VAE for enhanced depth and pose estimation in endoscopy, addressing generalizability issues of prior methods.

Findings

Outperforms existing self-supervised methods on endoscopic datasets

Improves depth prediction realism and robustness

Enhances pose estimation stability and accuracy

Abstract

Accurate 3D mapping in endoscopy enables quantitative, holistic lesion characterization within the gastrointestinal (GI) tract, requiring reliable depth and pose estimation. However, endoscopy systems are monocular, and existing methods relying on synthetic datasets or complex models often lack generalizability in challenging endoscopic conditions. We propose a robust self-supervised monocular depth and pose estimation framework that incorporates a Generative Latent Bank and a Variational Autoencoder (VAE). The Generative Latent Bank leverages extensive depth scenes from natural images to condition the depth network, enhancing realism and robustness of depth predictions through latent feature priors. For pose estimation, we reformulate it within a VAE framework, treating pose transitions as latent variables to regularize scale, stabilize z-axis prominence, and improve x-y sensitivity. This dual refinement pipeline enables accurate depth and pose predictions, effectively addressing the GI tract's complex textures and lighting. Extensive evaluations on SimCol and EndoSLAM datasets confirm our framework's superior performance over published self-supervised methods in endoscopic depth and pose estimation.