SliceMatch: Geometry-guided Aggregation for Cross-View Pose Estimation

TL;DR

SliceMatch introduces a geometry-guided feature aggregation method for cross-view camera pose estimation, significantly reducing localization error and enabling real-time performance by leveraging cross-view attention and geometric pooling.

Contribution

The paper presents a novel aerial feature aggregator with cross-view attention and geometric pooling, improving accuracy and efficiency in cross-view pose estimation.

Findings

19% lower median localization error on VIGOR with VGG16

50% lower error with ResNet50 backbone

Achieves 150 frames per second in real-time applications

Abstract

This work addresses cross-view camera pose estimation, i.e., determining the 3-Degrees-of-Freedom camera pose of a given ground-level image w.r.t. an aerial image of the local area. We propose SliceMatch, which consists of ground and aerial feature extractors, feature aggregators, and a pose predictor. The feature extractors extract dense features from the ground and aerial images. Given a set of candidate camera poses, the feature aggregators construct a single ground descriptor and a set of pose-dependent aerial descriptors. Notably, our novel aerial feature aggregator has a cross-view attention module for ground-view guided aerial feature selection and utilizes the geometric projection of the ground camera's viewing frustum on the aerial image to pool features. The efficient construction of aerial descriptors is achieved using precomputed masks. SliceMatch is trained using contrastive learning and pose estimation is formulated as a similarity comparison between the ground descriptor and the aerial descriptors. Compared to the state-of-the-art, SliceMatch achieves a 19% lower median localization error on the VIGOR benchmark using the same VGG16 backbone at 150 frames per second, and a 50% lower error when using a ResNet50 backbone.