MAG-VLAQ: Multi-modal Aerial-Ground Query Aggregation for Cross-View Place Recognition

TL;DR

MAG-VLAQ introduces a foundation-model-enhanced framework for multi-modal aerial-ground place recognition, effectively aligning heterogeneous visual and geometric data to improve cross-view matching accuracy.

Contribution

It proposes ODE-conditioned VLAQ, a novel fusion method that dynamically adapts query centers based on fused multi-modal information, enhancing retrieval performance.

Findings

Nearly doubles state-of-the-art on KITTI360-AG with 61.1% Recall@1.

Effectively aligns multi-modal data for improved cross-view recognition.

Validated on KITTI360-AG and nuScenes-AG datasets.

Abstract

Multi-modal cross-view place recognition remains a fundamental challenge in computer vision and robotics due to the severe viewpoint, modality, and spatial-structure discrepancies between ground observations and aerial references. To address this challenge, we present MAG-VLAQ, a foundation-model-enhanced query aggregation framework for multi-modal aerial-ground cross-view place recognition. Specifically, our approach leverages pre-trained foundation models to extract dense visual tokens from both ground and aerial images, as well as expressive geometric tokens from ground LiDAR observations. These heterogeneous tokens are then projected into a shared embedding space for cross-modal alignment and fusion. As our main contribution, we propose ODE-conditioned VLAQ, which tightly couples neural ordinary differential equations (ODE)-based RGB-LiDAR fusion with vectors of locally aggregated queries (VLAQ). In this design, the VLAQ query centers are dynamically adapted according to the fused multi-modal state. This mechanism allows the final global descriptor to preserve globally learned retrieval prototypes while remaining responsive to scene-specific visual and geometric evidence, significantly improving aerial-ground matching. Extensive experiments on KITTI360-AG and nuScenes-AG validate the effectiveness of our proposed MAG-VLAQ. Notably, on KITTI360-AG, our MAG-VLAQ nearly doubles the state-of-the-art performance, achieving 61.1 Recall@1 in the satellite setting, compared with 34.5 from the closest competing approach.