Seeing through Satellite Images at Street Views

TL;DR

This paper introduces Sat2Density++, a neural network approach for synthesizing photorealistic street-view panoramas from satellite images, effectively handling large viewpoint changes and street-specific elements.

Contribution

The paper proposes a novel neural radiance field model that incorporates street-view specific features to improve satellite-to-street panorama synthesis.

Findings

Capable of rendering photorealistic street-view panoramas

Maintains consistency across multiple views

Faithfully reflects satellite images

Abstract

This paper studies the task of SatStreet-view synthesis, which aims to render photorealistic street-view panorama images and videos given any satellite image and specified camera positions or trajectories. We formulate to learn neural radiance field from paired images captured from satellite and street viewpoints, which comes to be a challenging learning problem due to the sparse-view natural and the extremely-large viewpoint changes between satellite and street-view images. We tackle the challenges based on a task-specific observation that street-view specific elements, including the sky and illumination effects are only visible in street-view panoramas, and present a novel approach Sat2Density++ to accomplish the goal of photo-realistic street-view panoramas rendering by modeling these street-view specific in neural networks. In the experiments, our method is testified on both urban and suburban scene datasets, demonstrating that Sat2Density++ is capable of rendering photorealistic street-view panoramas that are consistent across multiple views and faithful to the satellite image.