Sat2Vid: Street-view Panoramic Video Synthesis from a Single Satellite Image

TL;DR

This paper introduces Sat2Vid, a novel method that synthesizes realistic, temporally, and geometrically consistent street-view panoramic videos from a single satellite image, leveraging 3D scene modeling and a cascaded neural network architecture.

Contribution

It is the first approach to generate street-view videos from satellite images, explicitly modeling 3D scene structure for consistent and realistic video synthesis.

Findings

Outperforms existing methods in realism and temporal consistency

Successfully synthesizes cross-view videos from satellite images

Demonstrates superior qualitative and quantitative results

Abstract

We present a novel method for synthesizing both temporally and geometrically consistent street-view panoramic video from a single satellite image and camera trajectory. Existing cross-view synthesis approaches focus on images, while video synthesis in such a case has not yet received enough attention. For geometrical and temporal consistency, our approach explicitly creates a 3D point cloud representation of the scene and maintains dense 3D-2D correspondences across frames that reflect the geometric scene configuration inferred from the satellite view. As for synthesis in the 3D space, we implement a cascaded network architecture with two hourglass modules to generate point-wise coarse and fine features from semantics and per-class latent vectors, followed by projection to frames and an upsampling module to obtain the final realistic video. By leveraging computed correspondences, the produced street-view video frames adhere to the 3D geometric scene structure and maintain temporal consistency. Qualitative and quantitative experiments demonstrate superior results compared to other state-of-the-art synthesis approaches that either lack temporal consistency or realistic appearance. To the best of our knowledge, our work is the first one to synthesize cross-view images to video.