Sky2Ground: A Benchmark for Site Modeling under Varying Altitude

TL;DR

Sky2Ground introduces a diverse dataset for evaluating camera localization and reconstruction across varying altitudes, and proposes SkyNet to improve multi-view consistency, outperforming existing methods.

Contribution

The paper presents Sky2Ground, a comprehensive multi-altitude dataset, and SkyNet, a novel model that enhances cross-view consistency with curriculum training, advancing large-scale 3D perception.

Findings

Sky2Ground dataset covers 51 sites with thousands of images at different altitudes.

State-of-the-art pose estimation models perform worse with satellite imagery.

SkyNet improves multi-view alignment, outperforming existing methods by 9.6% and 18.1%.

Abstract

We introduce Sky2Ground, a three-view dataset designed for varying altitude camera localization, correspondence learning, and reconstruction. The dataset combines structured synthetic imagery with real, in-the-wild images, providing both controlled multi-view geometry and realistic scene noise. Each of the 51 sites contains thousands of satellite, aerial, and ground images spanning wide altitude ranges and nearly orthogonal viewing angles, enabling rigorous evaluation across global-to-local contexts. We benchmark state of the art pose estimation models, including MASt3R, DUSt3R, Map Anything, and VGGT, and observe that the use of satellite imagery often degrades performance, highlighting the challenges under large altitude variations. We also examine reconstruction methods, highlighting the challenges introduced by sparse geometric overlap, varying perspectives, and the use of real imagery, which often introduces noise and reduces rendering quality. To address some of these challenges, we propose SkyNet, a model which enhances cross-view consistency when incorporating satellite imagery with a curriculum-based training strategy to progressively incorporate more satellite views. SkyNet significantly strengthens multi-view alignment and outperforms existing methods by 9.6% on RRA@5 and 18.1% on RTA@5 in terms of absolute performance. Sky2Ground and SkyNet together establish a comprehensive testbed and baseline for advancing large-scale, multi-altitude 3D perception and generalizable camera localization. Code and models will be released publicly for future research.Project page: https://sky2ground2026.github.io/sky2ground/