LiDAR-3DGS: LiDAR Reinforced 3D Gaussian Splatting for Multimodal Radiance Field Rendering

TL;DR

LiDAR-3DGS enhances 3D Gaussian Splatting for radiance field rendering by integrating LiDAR point clouds, significantly improving model detail and accuracy for engineering applications without altering the core algorithm.

Contribution

The paper introduces a novel method to incorporate LiDAR data into 3D Gaussian Splatting, boosting detail capture and model quality in radiance field rendering.

Findings

7.064% increase in PSNR at 30k iterations

0.565% increase in SSIM at 30k iterations

Modest improvements using commercial-grade LiDAR

Abstract

In this paper, we explore the capabilities of multimodal inputs to 3D Gaussian Splatting (3DGS) based Radiance Field Rendering. We present LiDAR-3DGS, a novel method of reinforcing 3DGS inputs with LiDAR generated point clouds to significantly improve the accuracy and detail of 3D models. We demonstrate a systematic approach of LiDAR reinforcement to 3DGS to enable capturing of important features such as bolts, apertures, and other details that are often missed by image-based features alone. These details are crucial for engineering applications such as remote monitoring and maintenance. Without modifying the underlying 3DGS algorithm, we demonstrate that even a modest addition of LiDAR generated point cloud significantly enhances the perceptual quality of the models. At 30k iterations, the model generated by our method resulted in an increase of 7.064% in PSNR and 0.565% in SSIM, respectively. Since the LiDAR used in this research was a commonly used commercial-grade device, the improvements observed were modest and can be further enhanced with higher-grade LiDAR systems. Additionally, these improvements can be supplementary to other derivative works of Radiance Field Rendering and also provide a new insight for future LiDAR and computer vision integrated modeling.