EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images

TL;DR

EndoUFM is an innovative unsupervised framework that leverages dual foundation models and adaptive fine-tuning to significantly improve monocular depth estimation in endoscopic images, aiding surgical navigation.

Contribution

The paper introduces EndoUFM, a novel unsupervised depth estimation method that integrates dual foundation models with RVLoRA and Res-DSC for enhanced performance in endoscopic scenes.

Findings

Achieves state-of-the-art accuracy on multiple datasets

Maintains efficient model size for real-time applications

Enhances depth perception for surgical navigation

Abstract

Depth estimation is a foundational component for 3D reconstruction in minimally invasive endoscopic surgeries. However, existing monocular depth estimation techniques often exhibit limited performance to the varying illumination and complex textures of the surgical environment. While powerful visual foundation models offer a promising solution, their training on natural images leads to significant domain adaptability limitations and semantic perception deficiencies when applied to endoscopy. In this study, we introduce EndoUFM, an unsupervised monocular depth estimation framework that innovatively integrating dual foundation models for surgical scenes, which enhance the depth estimation performance by leveraging the powerful pre-learned priors. The framework features a novel adaptive fine-tuning strategy that incorporates Random Vector Low-Rank Adaptation (RVLoRA) to enhance model adaptability, and a Residual block based on Depthwise Separable Convolution (Res-DSC) to improve the capture of fine-grained local features. Furthermore, we design a mask-guided smoothness loss to enforce depth consistency within anatomical tissue structures. Extensive experiments on the SCARED, Hamlyn, SERV-CT, and EndoNeRF datasets confirm that our method achieves state-of-the-art performance while maintaining an efficient model size. This work contributes to augmenting surgeons' spatial perception during minimally invasive procedures, thereby enhancing surgical precision and safety, with crucial implications for augmented reality and navigation systems.