CRISTAL: Real-time Camera Registration in Static LiDAR Scans using Neural Rendering

TL;DR

This paper presents a real-time camera localization method within static LiDAR scans using neural rendering to improve accuracy and eliminate drift, outperforming existing SLAM systems on benchmark datasets.

Contribution

Introduces a novel real-time camera registration approach combining neural rendering with LiDAR point clouds, enabling drift-free and metric scale accurate localization.

Findings

Outperforms existing SLAM pipelines on ScanNet++ dataset

Provides drift-free camera tracking with correct metric scale

Offers two real-time variants: Online Render and Match, and Prebuild and Localize

Abstract

Accurate camera localization is crucial for robotics and Extended Reality (XR), enabling reliable navigation and alignment of virtual and real content. Existing visual methods often suffer from drift, scale ambiguity, and depend on fiducials or loop closure. This work introduces a real-time method for localizing a camera within a pre-captured, highly accurate colored LiDAR point cloud. By rendering synthetic views from this cloud, 2D-3D correspondences are established between live frames and the point cloud. A neural rendering technique narrows the domain gap between synthetic and real images, reducing occlusion and background artifacts to improve feature matching. The result is drift-free camera tracking with correct metric scale in the global LiDAR coordinate system. Two real-time variants are presented: Online Render and Match, and Prebuild and Localize. We demonstrate improved results on the ScanNet++ dataset and outperform existing SLAM pipelines.