SAGE: SLAM with Appearance and Geometry Prior for Endoscopy

TL;DR

SAGE introduces a real-time SLAM system for endoscopy that combines learned appearance and geometry priors with factor graph optimization, improving robustness and generalization in challenging conditions.

Contribution

The paper presents a novel end-to-end differentiable SLAM system that integrates learned appearance and geometry priors for endoscopic applications.

Findings

Robust handling of texture scarcity and illumination changes

Good generalization to unseen endoscopes and subjects

Outperforms state-of-the-art feature-based SLAM systems

Abstract

In endoscopy, many applications (e.g., surgical navigation) would benefit from a real-time method that can simultaneously track the endoscope and reconstruct the dense 3D geometry of the observed anatomy from a monocular endoscopic video. To this end, we develop a Simultaneous Localization and Mapping system by combining the learning-based appearance and optimizable geometry priors and factor graph optimization. The appearance and geometry priors are explicitly learned in an end-to-end differentiable training pipeline to master the task of pair-wise image alignment, one of the core components of the SLAM system. In our experiments, the proposed SLAM system is shown to robustly handle the challenges of texture scarceness and illumination variation that are commonly seen in endoscopy. The system generalizes well to unseen endoscopes and subjects and performs favorably compared with a state-of-the-art feature-based SLAM system. The code repository is available at https://github.com/lppllppl920/SAGE-SLAM.git.