Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images

TL;DR

Cross-View Splatter is a feed-forward approach that combines satellite and ground imagery to improve outdoor scene reconstruction and novel-view synthesis using georeferenced data.

Contribution

It introduces a method that fuses satellite and ground images in a unified 3D frame, enhancing scene coverage and view synthesis capabilities.

Findings

Improved scene coverage over ground-only methods

Effective fusion of satellite and ground imagery

Competitive results on a new georeferenced imagery benchmark

Abstract

We present Cross-View Splatter, a feed-forward method that predicts pixel-aligned Gaussian splats for outdoor scenes captured at ground level AND by satellite. Faithful reconstructions require good camera coverage, but ground imagery is time-consuming and hard to capture at scale for large outdoor scenes. Fortunately, satellite imagery can provide a global geometric prior that is easy to access via public APIs. Cross-View Splatter fuses orthorectified satellite views with GPS-tagged ground photos to predict Gaussian splats in a unified 3D coordinate frame. By aligning ground and bird's-eye feature representations, our model improves scene coverage and novel-view synthesis, compared to ground imagery alone. We train on curated georeferenced datasets and paired satellite-terrain data, mined from open mapping services. We evaluate our method on a new benchmark for novel-view synthesis with georeferenced imagery allowing comparison to prior state-of-the-art methods. Our code and data preparation will be available at https://nianticspatial.github.io/cross-view-splatter/.