EndoSERV: A Vision-based Endoluminal Robot Navigation System

TL;DR

EndoSERV introduces a novel vision-based localization system for endoluminal robots, effectively addressing tissue deformation and landmark scarcity to improve navigation accuracy in complex luminal structures.

Contribution

The paper proposes EndoSERV, a new localization method combining segmentation and real-to-virtual mapping, with offline pretraining and online adaptation for robust in vivo navigation.

Findings

Effective in complex luminal structures

Operates without real pose labels

Validated on public and clinical datasets

Abstract

Robot-assisted endoluminal procedures are increasingly used for early cancer intervention. However, the intricate, narrow and tortuous pathways within the luminal anatomy pose substantial difficulties for robot navigation. Vision-based navigation offers a promising solution, but existing localization approaches are error-prone due to tissue deformation, in vivo artifacts and a lack of distinctive landmarks for consistent localization. This paper presents a novel EndoSERV localization method to address these challenges. It includes two main parts, \textit{i.e.}, \textbf{SE}gment-to-structure and \textbf{R}eal-to-\textbf{V}irtual mapping, and hence the name. For long-range and complex luminal structures, we divide them into smaller sub-segments and estimate the odometry independently. To cater for label insufficiency, an efficient transfer technique maps real image features to the virtual domain to use virtual pose ground truth. The training phases of EndoSERV include an offline pretraining to extract texture-agnostic features, and an online phase that adapts to real-world conditions. Extensive experiments based on both public and clinical datasets have been performed to demonstrate the effectiveness of the method even without any real pose labels.