GSPR: Multimodal Place Recognition Using 3D Gaussian Splatting for Autonomous Driving

TL;DR

This paper introduces GSPR, a multimodal place recognition method using 3D Gaussian Splatting that effectively combines RGB images and LiDAR data into a unified scene representation, achieving state-of-the-art results in autonomous driving environments.

Contribution

It proposes a novel 3D Gaussian Splatting-based network for explicit multimodal fusion, improving interpretability and performance in place recognition tasks.

Findings

Achieves state-of-the-art performance on three datasets.

Effectively leverages both camera and LiDAR data.

Demonstrates strong generalization ability.

Abstract

Place recognition is a crucial component that enables autonomous vehicles to obtain localization results in GPS-denied environments. In recent years, multimodal place recognition methods have gained increasing attention. They overcome the weaknesses of unimodal sensor systems by leveraging complementary information from different modalities. However, most existing methods explore cross-modality correlations through feature-level or descriptor-level fusion, suffering from a lack of interpretability. Conversely, the recently proposed 3D Gaussian Splatting provides a new perspective on multimodal fusion by harmonizing different modalities into an explicit scene representation. In this paper, we propose a 3D Gaussian Splatting-based multimodal place recognition network dubbed GSPR. It explicitly combines multi-view RGB images and LiDAR point clouds into a spatio-temporally unified scene representation with the proposed Multimodal Gaussian Splatting. A network composed of 3D graph convolution and transformer is designed to extract spatio-temporal features and global descriptors from the Gaussian scenes for place recognition. Extensive evaluations on three datasets demonstrate that our method can effectively leverage complementary strengths of both multi-view cameras and LiDAR, achieving SOTA place recognition performance while maintaining solid generalization ability. Our open-source code will be released at https://github.com/QiZS-BIT/GSPR.