DD-VNB: A Depth-based Dual-Loop Framework for Real-time Visually Navigated Bronchoscopy

TL;DR

This paper introduces DD-VNB, a depth-based dual-loop framework for real-time bronchoscopic navigation that generalizes across patients without retraining, achieving high accuracy and speed in localization.

Contribution

The study presents a novel depth estimation and localization framework that combines view synthesis, cycle adversarial architecture, and fast ego-motion estimation for real-time bronchoscopic navigation.

Findings

Depth estimation outperforms state-of-the-art methods.

Localization accuracy with ATE of 4.7 mm in phantom and 6.49 mm in patient data.

Achieves near video frame-rate processing without retraining.

Abstract

Real-time 6 DOF localization of bronchoscopes is crucial for enhancing intervention quality. However, current vision-based technologies struggle to balance between generalization to unseen data and computational speed. In this study, we propose a Depth-based Dual-Loop framework for real-time Visually Navigated Bronchoscopy (DD-VNB) that can generalize across patient cases without the need of re-training. The DD-VNB framework integrates two key modules: depth estimation and dual-loop localization. To address the domain gap among patients, we propose a knowledge-embedded depth estimation network that maps endoscope frames to depth, ensuring generalization by eliminating patient-specific textures. The network embeds view synthesis knowledge into a cycle adversarial architecture for scale-constrained monocular depth estimation. For real-time performance, our localization module embeds a fast ego-motion estimation network into the loop of depth registration. The ego-motion inference network estimates the pose change of the bronchoscope in high frequency while depth registration against the pre-operative 3D model provides absolute pose periodically. Specifically, the relative pose changes are fed into the registration process as the initial guess to boost its accuracy and speed. Experiments on phantom and in-vivo data from patients demonstrate the effectiveness of our framework: 1) monocular depth estimation outperforms SOTA, 2) localization achieves an accuracy of Absolute Tracking Error (ATE) of 4.7 $\pm$ 3.17 mm in phantom and 6.49 $\pm$ 3.88 mm in patient data, 3) with a frame-rate approaching video capture speed, 4) without the necessity of case-wise network retraining. The framework's superior speed and accuracy demonstrate its promising clinical potential for real-time bronchoscopic navigation.