SEGS-SLAM: Structure-enhanced 3D Gaussian Splatting SLAM with Appearance Embedding

TL;DR

SEGS-SLAM introduces a structure-enhanced 3D Gaussian Splatting SLAM method that leverages structured point clouds and appearance embedding to achieve photorealistic mapping with significant quality improvements.

Contribution

It proposes a novel structure-enhanced mapping framework and appearance-from-motion embedding, improving 3D Gaussian initialization and modeling appearance variations in SLAM.

Findings

Outperforms SOTA methods in mapping quality

Achieves 19.86% PSNR improvement over MonoGS on TUM RGB-D

Demonstrates effectiveness across monocular, stereo, and RGB-D datasets

Abstract

3D Gaussian splatting (3D-GS) has recently revolutionized novel view synthesis in the simultaneous localization and mapping (SLAM) problem. However, most existing algorithms fail to fully capture the underlying structure, resulting in structural inconsistency. Additionally, they struggle with abrupt appearance variations, leading to inconsistent visual quality. To address these problems, we propose SEGS-SLAM, a structure-enhanced 3D Gaussian Splatting SLAM, which achieves high-quality photorealistic mapping. Our main contributions are two-fold. First, we propose a structure-enhanced photorealistic mapping (SEPM) framework that, for the first time, leverages highly structured point cloud to initialize structured 3D Gaussians, leading to significant improvements in rendering quality. Second, we propose Appearance-from-Motion embedding (AfME), enabling 3D Gaussians to better model image appearance variations across different camera poses. Extensive experiments on monocular, stereo, and RGB-D datasets demonstrate that SEGS-SLAM significantly outperforms state-of-the-art (SOTA) methods in photorealistic mapping quality, e.g., an improvement of $19.86\%$ in PSNR over MonoGS on the TUM RGB-D dataset for monocular cameras. The project page is available at https://segs-slam.github.io/.