SAGE: SLAM with Appearance and Geometry Prior for Endoscopy
Xingtong Liu, Zhaoshuo Li, Masaru Ishii, Gregory D. Hager, Russell H., Taylor, Mathias Unberath

TL;DR
SAGE introduces a real-time SLAM system for endoscopy that combines learned appearance and geometry priors with factor graph optimization, improving robustness and generalization in challenging conditions.
Contribution
The paper presents a novel end-to-end differentiable SLAM system that integrates learned appearance and geometry priors for endoscopic applications.
Findings
Robust handling of texture scarcity and illumination changes
Good generalization to unseen endoscopes and subjects
Outperforms state-of-the-art feature-based SLAM systems
Abstract
In endoscopy, many applications (e.g., surgical navigation) would benefit from a real-time method that can simultaneously track the endoscope and reconstruct the dense 3D geometry of the observed anatomy from a monocular endoscopic video. To this end, we develop a Simultaneous Localization and Mapping system by combining the learning-based appearance and optimizable geometry priors and factor graph optimization. The appearance and geometry priors are explicitly learned in an end-to-end differentiable training pipeline to master the task of pair-wise image alignment, one of the core components of the SLAM system. In our experiments, the proposed SLAM system is shown to robustly handle the challenges of texture scarceness and illumination variation that are commonly seen in endoscopy. The system generalizes well to unseen endoscopes and subjects and performs favorably compared with a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Image and Video Retrieval Techniques · Robotics and Sensor-Based Localization · Multimodal Machine Learning Applications
