ACE-SLAM: Scene Coordinate Regression for Neural Implicit Real-Time SLAM

TL;DR

ACE-SLAM introduces a real-time neural RGB-D SLAM system using Scene Coordinate Regression, enabling efficient, privacy-preserving 3D mapping and fast relocalization in dynamic environments.

Contribution

It is the first to employ SCR as the core implicit map in a neural SLAM pipeline, achieving real-time performance with a novel SCR architecture.

Findings

Achieves strict real-time operation in neural implicit RGB-D SLAM.

Provides efficient, low-memory 3D map representations.

Demonstrates competitive results on synthetic and real-world benchmarks.

Abstract

We present a novel neural RGB-D Simultaneous Localization And Mapping (SLAM) system that learns an implicit map of the scene in real time. For the first time, we explore the use of Scene Coordinate Regression (SCR) as the core implicit map representation in a neural SLAM pipeline, a paradigm that trains a lightweight network to directly map 2D image features to 3D global coordinates. SCR networks provide efficient, low-memory 3D map representations, enable extremely fast relocalization, and inherently preserve privacy, making them particularly suitable for neural implicit SLAM. Our system is the first one to achieve strict real-time in neural implicit RGB-D SLAM by relying on a SCR-based representation. We introduce a novel SCR architecture specifically tailored for this purpose and detail the critical design choices required to integrate SCR into a live SLAM pipeline. The resulting framework is simple yet flexible, seamlessly supporting both sparse and dense features, and operates reliably in dynamic environments without special adaptation. We evaluate our approach on established synthetic and real-world benchmarks, demonstrating competitive performance against the state of the art. Project Page: https://github.com/ialzugaray/ace-slam