SMGeo: Cross-View Object Geo-Localization with Grid-Level Mixture-of-Experts

TL;DR

SMGeo is an innovative transformer-based model that enables real-time, interactive cross-view object geo-localization by adaptively leveraging grid-level experts, significantly outperforming previous methods in accuracy.

Contribution

The paper introduces SMGeo, a novel end-to-end transformer model with grid-level sparse Mixture-of-Experts for improved cross-view geo-localization accuracy and real-time interactive capabilities.

Findings

Achieves state-of-the-art accuracy on drone-to-satellite localization tasks.

Outperforms existing methods like DetGeo in key metrics.

Demonstrates effectiveness of grid-level MoE and anchor-free detection in this context.

Abstract

Cross-view object Geo-localization aims to precisely pinpoint the same object across large-scale satellite imagery based on drone images. Due to significant differences in viewpoint and scale, coupled with complex background interference, traditional multi-stage "retrieval-matching" pipelines are prone to cumulative errors. To address this, we present SMGeo, a promptable end-to-end transformer-based model for object Geo-localization. This model supports click prompting and can output object Geo-localization in real time when prompted to allow for interactive use. The model employs a fully transformer-based architecture, utilizing a Swin-Transformer for joint feature encoding of both drone and satellite imagery and an anchor-free transformer detection head for coordinate regression. In order to better capture both inter-modal and intra-view dependencies, we introduce a grid-level sparse Mixture-of-Experts (GMoE) into the cross-view encoder, allowing it to adaptively activate specialized experts according to the content, scale and source of each grid. We also employ an anchor-free detection head for coordinate regression, directly predicting object locations via heat-map supervision in the reference images. This approach avoids scale bias and matching complexity introduced by predefined anchor boxes. On the drone-to-satellite task, SMGeo achieves leading performance in accuracy at IoU=0.25 and mIoU metrics (e.g., 87.51%, 62.50%, and 61.45% in the test set, respectively), significantly outperforming representative methods such as DetGeo (61.97%, 57.66%, and 54.05%, respectively). Ablation studies demonstrate complementary gains from shared encoding, query-guided fusion, and grid-level sparse mixture-of-experts.