Gaussian-LIC2: LiDAR-Inertial-Camera Gaussian Splatting SLAM

TL;DR

This paper introduces a real-time, photo-realistic LiDAR-Inertial-Camera SLAM system that combines Gaussian splatting with depth completion and trajectory optimization, enabling high-quality mapping and view synthesis.

Contribution

The novel system integrates Gaussian splatting with multi-sensor data fusion, depth completion, and trajectory optimization for improved SLAM performance and visual quality.

Findings

Achieves real-time, photo-realistic 3D mapping with high geometric accuracy.

Enhances pose estimation robustness under LiDAR degradation.

Provides a new dataset for evaluating multi-sensor SLAM systems.

Abstract

This paper presents the first photo-realistic LiDAR-Inertial-Camera Gaussian Splatting SLAM system that simultaneously addresses visual quality, geometric accuracy, and real-time performance. The proposed method performs robust and accurate pose estimation within a continuous-time trajectory optimization framework, while incrementally reconstructing a 3D Gaussian map using camera and LiDAR data, all in real time. The resulting map enables high-quality, real-time novel view rendering of both RGB images and depth maps. To effectively address under-reconstruction in regions not covered by the LiDAR, we employ a lightweight zero-shot depth model that synergistically combines RGB appearance cues with sparse LiDAR measurements to generate dense depth maps. The depth completion enables reliable Gaussian initialization in LiDAR-blind areas, significantly improving system applicability for sparse LiDAR sensors. To enhance geometric accuracy, we use sparse but precise LiDAR depths to supervise Gaussian map optimization and accelerate it with carefully designed CUDA-accelerated strategies. Furthermore, we explore how the incrementally reconstructed Gaussian map can improve the robustness of odometry. By tightly incorporating photometric constraints from the Gaussian map into the continuous-time factor graph optimization, we demonstrate improved pose estimation under LiDAR degradation scenarios. We also showcase downstream applications via extending our elaborate system, including video frame interpolation and fast 3D mesh extraction. To support rigorous evaluation, we construct a dedicated LiDAR-Inertial-Camera dataset featuring ground-truth poses, depth maps, and extrapolated trajectories for assessing out-of-sequence novel view synthesis. Both the dataset and code will be made publicly available on project page https://xingxingzuo.github.io/gaussian_lic2.