SLAM assisted 3D tracking system for laparoscopic surgery

TL;DR

This paper introduces a real-time monocular 3D tracking system for laparoscopic surgery that enhances accuracy and robustness in locating internal organs by integrating geometric priors and segmentation strategies within an augmented reality framework.

Contribution

It presents a novel modification of ORB-SLAM2 for organ tracking using shape priors and pseudo-segmentation, improving robustness in challenging surgical scenarios.

Findings

Robust 3D tracking in vivo and ex vivo tests

Effective handling of fast motion and partial visibility

Improved accuracy over traditional methods

Abstract

A major limitation of minimally invasive surgery is the difficulty in accurately locating the internal anatomical structures of the target organ due to the lack of tactile feedback and transparency. Augmented reality (AR) offers a promising solution to overcome this challenge. Numerous studies have shown that combining learning-based and geometric methods can achieve accurate preoperative and intraoperative data registration. This work proposes a real-time monocular 3D tracking algorithm for post-registration tasks. The ORB-SLAM2 framework is adopted and modified for prior-based 3D tracking. The primitive 3D shape is used for fast initialization of the monocular SLAM. A pseudo-segmentation strategy is employed to separate the target organ from the background for tracking purposes, and the geometric prior of the 3D shape is incorporated as an additional constraint in the pose graph. Experiments from in-vivo and ex-vivo tests demonstrate that the proposed 3D tracking system provides robust 3D tracking and effectively handles typical challenges such as fast motion, out-of-field-of-view scenarios, partial visibility, and "organ-background" relative motion.