SLC$^2$-SLAM: Semantic-guided Loop Closure using Shared Latent Code for NeRF SLAM

TL;DR

SLC$^2$-SLAM introduces a semantic-guided loop closure method for NeRF SLAM that leverages shared latent codes for improved loop detection and scene reconstruction, significantly outperforming existing approaches on benchmark datasets.

Contribution

The paper presents a novel loop closure detection method using shared latent codes and semantic information in NeRF SLAM, enhancing accuracy and robustness.

Findings

Outperforms NetVLAD and ORB with Bag-of-Words in loop detection

Improves tracking and reconstruction in large scenes with many loops

Demonstrates superior performance on Replica and ScanNet datasets

Abstract

Targeting the notorious cumulative drift errors in NeRF SLAM, we propose a Semantic-guided Loop Closure using Shared Latent Code, dubbed SLC$^2$-SLAM. We argue that latent codes stored in many NeRF SLAM systems are not fully exploited, as they are only used for better reconstruction. In this paper, we propose a simple yet effective way to detect potential loops using the same latent codes as local features. To further improve the loop detection performance, we use the semantic information, which are also decoded from the same latent codes to guide the aggregation of local features. Finally, with the potential loops detected, we close them with a graph optimization followed by bundle adjustment to refine both the estimated poses and the reconstructed scene. To evaluate the performance of our SLC$^2$-SLAM, we conduct extensive experiments on Replica and ScanNet datasets. Our proposed semantic-guided loop closure significantly outperforms the pre-trained NetVLAD and ORB combined with Bag-of-Words, which are used in all the other NeRF SLAM with loop closure. As a result, our SLC$^2$-SLAM also demonstrated better tracking and reconstruction performance, especially in larger scenes with more loops, like ScanNet.