FeatureSLAM: Feature-enriched 3D gaussian splatting SLAM in real time

TL;DR

FeatureSLAM is a real-time SLAM system that combines camera tracking with photorealistic, feature-enriched 3D Gaussian Splatting maps, enabling semantic understanding and open-set segmentation for downstream tasks.

Contribution

It introduces dense feature rasterization into 3D Gaussian Splatting for real-time, semantic SLAM, surpassing previous methods in accuracy and enabling new downstream applications.

Findings

Achieves 9% lower pose error compared to recent baselines.

Improves mapping accuracy by 8% over fixed-set SLAM methods.

Maintains real-time performance with state-of-the-art tracking and mapping quality.

Abstract

We present a real-time tracking SLAM system that unifies efficient camera tracking with photorealistic feature-enriched mapping using 3D Gaussian Splatting (3DGS). Our main contribution is integrating dense feature rasterization into the novel-view synthesis, aligned with a visual foundation model. This yields strong semantics, going beyond basic RGB-D input, aiding both tracking and mapping accuracy. Unlike previous semantic SLAM approaches (which embed pre-defined class labels) FeatureSLAM enables entirely new downstream tasks via free-viewpoint, open-set segmentation. Across standard benchmarks, our method achieves real-time tracking, on par with state-of-the-art systems while improving tracking stability and map fidelity without prohibitive compute. Quantitatively, we obtain 9\% lower pose error and 8\% higher mapping accuracy compared to recent fixed-set SLAM baselines. Our results confirm that real-time feature-embedded SLAM, is not only valuable for enabling new downstream applications. It also improves the performance of the underlying tracking and mapping subsystems, providing semantic and language masking results that are on-par with offline 3DGS models, alongside state-of-the-art tracking, depth and RGB rendering.