Magnetic-Visual Sensor Fusion-based Dense 3D Reconstruction and Localization for Endoscopic Capsule Robots

TL;DR

This paper presents a real-time, dense 3D reconstruction and localization method for endoscopic capsule robots, combining magnetic and vision data to handle non-rigid deformations in gastrointestinal navigation.

Contribution

It introduces a novel intraoperative map fusion approach that integrates magnetic and vision-based localization with non-rigid deformation modeling for capsule endoscopy.

Findings

Achieved root mean square surface errors of 1.58 to 2.17 cm across tests.

Demonstrated effectiveness on four ex-vivo porcine stomach models.

Validated performance with different trajectories, speeds, and camera types.

Abstract

Reliable and real-time 3D reconstruction and localization functionality is a crucial prerequisite for the navigation of actively controlled capsule endoscopic robots as an emerging, minimally invasive diagnostic and therapeutic technology for use in the gastrointestinal (GI) tract. In this study, we propose a fully dense, non-rigidly deformable, strictly real-time, intraoperative map fusion approach for actively controlled endoscopic capsule robot applications which combines magnetic and vision-based localization, with non-rigid deformations based frame-to-model map fusion. The performance of the proposed method is demonstrated using four different ex-vivo porcine stomach models. Across different trajectories of varying speed and complexity, and four different endoscopic cameras, the root mean square surface reconstruction errors 1.58 to 2.17 cm.