UniMPR: A Unified Framework for Multimodal Place Recognition with Heterogeneous Sensor Configurations

TL;DR

UniMPR introduces a versatile, unified framework for multimodal place recognition that adapts to various sensor setups, maintains robustness with missing data, and generalizes across diverse environments using a polar BEV feature space.

Contribution

The paper presents UniMPR, a single model that seamlessly adapts to different sensor modalities and configurations, addressing key challenges in multimodal place recognition.

Findings

Achieves state-of-the-art results across seven datasets.

Demonstrates robustness with missing or degraded modalities.

Effectively generalizes across diverse sensor configurations.

Abstract

Place recognition is a critical component of autonomous vehicles and robotics, enabling global localization in GPS-denied environments. Recent advances have spurred significant interest in multimodal place recognition (MPR), which leverages complementary strengths of multiple modalities. Despite its potential, most existing MPR methods still face three key challenges: (1) dynamically adapting to various modality inputs within a unified framework, (2) maintaining robustness with missing or degraded modalities, and (3) generalizing across diverse sensor configurations and setups. In this paper, we propose UniMPR, a unified framework for multimodal place recognition. Using only one trained model, it can seamlessly adapt to any combination of common perceptual modalities (e.g., camera, LiDAR, radar). To tackle the data heterogeneity, we unify all inputs within a polar BEV feature space. Subsequently, the polar BEVs are fed into a multi-branch network to exploit discriminative intra-model and inter-modal features from any modality combinations. To fully exploit the network's generalization capability and robustness, we construct a large-scale training set from multiple datasets and introduce an adaptive label assignment strategy for extensive pre-training. Experiments on seven datasets demonstrate that UniMPR achieves state-of-the-art performance under varying sensor configurations, modality combinations, and environmental conditions. Our code will be released at https://github.com/QiZS-BIT/UniMPR.