Query-Based Adaptive Aggregation for Multi-Dataset Joint Training Toward Universal Visual Place Recognition

TL;DR

This paper introduces Query-based Adaptive Aggregation (QAA), a novel feature aggregation method that enhances multi-dataset training for universal visual place recognition, improving generalization and performance across diverse datasets.

Contribution

QAA leverages learned queries as reference codebooks to improve feature aggregation in multi-dataset training, addressing dataset divergence issues without increasing complexity.

Findings

QAA outperforms state-of-the-art models in diverse datasets.

QAA achieves balanced generalization and high performance.

Ablation studies confirm QAA's effectiveness and scalability.

Abstract

Deep learning methods for Visual Place Recognition (VPR) have advanced significantly, largely driven by large-scale datasets. However, most existing approaches are trained on a single dataset, which can introduce dataset-specific inductive biases and limit model generalization. While multi-dataset joint training offers a promising solution for developing universal VPR models, divergences among training datasets can saturate the limited information capacity in feature aggregation layers, leading to suboptimal performance. To address these challenges, we propose Query-based Adaptive Aggregation (QAA), a novel feature aggregation technique that leverages learned queries as reference codebooks to effectively enhance information capacity without significant computational or parameter complexity. We show that computing the Cross-query Similarity (CS) between query-level image features and reference codebooks provides a simple yet effective way to generate robust descriptors. Our results demonstrate that QAA outperforms state-of-the-art models, achieving balanced generalization across diverse datasets while maintaining peak performance comparable to dataset-specific models. Ablation studies further explore QAA's mechanisms and scalability. Visualizations reveal that the learned queries exhibit diverse attention patterns across datasets. Project page: http://xjh19971.github.io/QAA.