Patch-NetVLAD+: Learned patch descriptor and weighted matching strategy for place recognition

TL;DR

Patch-NetVLAD+ enhances visual place recognition by learning patch descriptors and weighting critical local regions, significantly improving accuracy in challenging urban and indoor scenarios.

Contribution

Introduces Patch-NetVLAD+ with a fine-tuning triplet loss and weighted patch selection to improve local scene feature extraction for VPR.

Findings

Achieved up to 6.35% performance improvement over existing methods.

Effectively identifies and emphasizes less frequent local scene regions.

Demonstrated superior results on Pittsburgh30k and Tokyo247 datasets.

Abstract

Visual Place Recognition (VPR) in areas with similar scenes such as urban or indoor scenarios is a major challenge. Existing VPR methods using global descriptors have difficulty capturing local specific regions (LSR) in the scene and are therefore prone to localization confusion in such scenarios. As a result, finding the LSR that are critical for location recognition becomes key. To address this challenge, we introduced Patch-NetVLAD+, which was inspired by patch-based VPR researches. Our method proposed a fine-tuning strategy with triplet loss to make NetVLAD suitable for extracting patch-level descriptors. Moreover, unlike existing methods that treat all patches in an image equally, our method extracts patches of LSR, which present less frequently throughout the dataset, and makes them play an important role in VPR by assigning proper weights to them. Experiments on Pittsburgh30k and Tokyo247 datasets show that our approach achieved up to 6.35\% performance improvement than existing patch-based methods.