EmbodiedPlace: Learning Mixture-of-Features with Embodied Constraints for Visual Place Recognition

TL;DR

This paper introduces EmbodiedPlace, a novel re-ranking method for visual place recognition that refines global features using a mixture-of-features approach under embodied constraints, improving accuracy with minimal computational cost.

Contribution

It proposes a learning-based mixture-of-features re-ranking method for VPR that incorporates embodied constraints, enhancing state-of-the-art performance with low overhead.

Findings

Improves SOTA performance on public datasets.

Achieves 0.9% accuracy gain with minimal parameters.

Operates with only 10 microseconds per frame.

Abstract

Visual Place Recognition (VPR) is a scene-oriented image retrieval problem in computer vision in which re-ranking based on local features is commonly employed to improve performance. In robotics, VPR is also referred to as Loop Closure Detection, which emphasizes spatial-temporal verification within a sequence. However, designing local features specifically for VPR is impractical, and relying on motion sequences imposes limitations. Inspired by these observations, we propose a novel, simple re-ranking method that refines global features through a Mixture-of-Features (MoF) approach under embodied constraints. First, we analyze the practical feasibility of embodied constraints in VPR and categorize them according to existing datasets, which include GPS tags, sequential timestamps, local feature matching, and self-similarity matrices. We then propose a learning-based MoF weight-computation approach, utilizing a multi-metric loss function. Experiments demonstrate that our method improves the state-of-the-art (SOTA) performance on public datasets with minimal additional computational overhead. For instance, with only 25 KB of additional parameters and a processing time of 10 microseconds per frame, our method achieves a 0.9\% improvement over a DINOv2-based baseline performance on the Pitts-30k test set.