Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network

TL;DR

This paper introduces the Panorama-BEV Co-Retrieval Network for cross-view geolocalization, converting street panoramas into bird's-eye views to improve matching accuracy with satellite images, and presents a new dataset for evaluation.

Contribution

The paper proposes a novel network that combines BEV conversion with collaborative retrieval branches and introduces a realistic, challenging dataset for cross-view localization.

Findings

Outperforms existing methods on multiple datasets

Effectively bridges the gap between street view and satellite imagery

Provides a comprehensive evaluation with the new CVGlobal dataset

Abstract

Cross-view geolocalization identifies the geographic location of street view images by matching them with a georeferenced satellite database. Significant challenges arise due to the drastic appearance and geometry differences between views. In this paper, we propose a new approach for cross-view image geo-localization, i.e., the Panorama-BEV Co-Retrieval Network. Specifically, by utilizing the ground plane assumption and geometric relations, we convert street view panorama images into the BEV view, reducing the gap between street panoramas and satellite imagery. In the existing retrieval of street view panorama images and satellite images, we introduce BEV and satellite image retrieval branches for collaborative retrieval. By retaining the original street view retrieval branch, we overcome the limited perception range issue of BEV representation. Our network enables comprehensive perception of both the global layout and local details around the street view capture locations. Additionally, we introduce CVGlobal, a global cross-view dataset that is closer to real-world scenarios. This dataset adopts a more realistic setup, with street view directions not aligned with satellite images. CVGlobal also includes cross-regional, cross-temporal, and street view to map retrieval tests, enabling a comprehensive evaluation of algorithm performance. Our method excels in multiple tests on common cross-view datasets such as CVUSA, CVACT, VIGOR, and our newly introduced CVGlobal, surpassing the current state-of-the-art approaches. The code and datasets can be found at \url{https://github.com/yejy53/EP-BEV}.