UniLoc: Towards Universal Place Recognition Using Any Single Modality

TL;DR

UniLoc introduces a universal place recognition system capable of handling any single modality such as language, images, or point clouds, leveraging contrastive learning and a novel pooling method for improved cross-modal and uni-modal performance.

Contribution

It presents a novel universal place recognition framework that works across different modalities using hierarchical contrastive learning and a self-attention pooling module.

Findings

Achieves superior cross-modal place recognition performance.

Demonstrates competitive results in uni-modal scenarios.

Validates effectiveness on the KITTI-360 dataset.

Abstract

To date, most place recognition methods focus on single-modality retrieval. While they perform well in specific environments, cross-modal methods offer greater flexibility by allowing seamless switching between map and query sources. It also promises to reduce computation requirements by having a unified model, and achieving greater sample efficiency by sharing parameters. In this work, we develop a universal solution to place recognition, UniLoc, that works with any single query modality (natural language, image, or point cloud). UniLoc leverages recent advances in large-scale contrastive learning, and learns by matching hierarchically at two levels: instance-level matching and scene-level matching. Specifically, we propose a novel Self-Attention based Pooling (SAP) module to evaluate the importance of instance descriptors when aggregated into a place-level descriptor. Experiments on the KITTI-360 dataset demonstrate the benefits of cross-modality for place recognition, achieving superior performance in cross-modal settings and competitive results also for uni-modal scenarios. Our project page is publicly available at https://yan-xia.github.io/projects/UniLoc/.