TransVPR: Transformer-based place recognition with multi-level attention aggregation

TL;DR

TransVPR is a novel place recognition model using vision Transformers that effectively integrates multi-level attention to focus on task-relevant regions, achieving state-of-the-art results with efficient computation.

Contribution

It introduces a Transformer-based holistic place recognition approach that combines multi-level attention for improved accuracy and efficiency in complex scenes.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Maintains low computational time and storage.

Effectively filters task-relevant features using attention masks.

Abstract

Visual place recognition is a challenging task for applications such as autonomous driving navigation and mobile robot localization. Distracting elements presenting in complex scenes often lead to deviations in the perception of visual place. To address this problem, it is crucial to integrate information from only task-relevant regions into image representations. In this paper, we introduce a novel holistic place recognition model, TransVPR, based on vision Transformers. It benefits from the desirable property of the self-attention operation in Transformers which can naturally aggregate task-relevant features. Attentions from multiple levels of the Transformer, which focus on different regions of interest, are further combined to generate a global image representation. In addition, the output tokens from Transformer layers filtered by the fused attention mask are considered as key-patch descriptors, which are used to perform spatial matching to re-rank the candidates retrieved by the global image features. The whole model allows end-to-end training with a single objective and image-level supervision. TransVPR achieves state-of-the-art performance on several real-world benchmarks while maintaining low computational time and storage requirements.