Semantic-aware Network for Aerial-to-Ground Image Synthesis

TL;DR

This paper introduces a semantic-aware network that improves aerial-to-ground image synthesis by enhancing structural alignment and semantic understanding, leading to more realistic ground images from aerial inputs.

Contribution

It proposes a novel semantic-attentive feature transformation module and semantic-aware loss functions to better capture geographic structures and object details.

Findings

Outperforms previous methods in qualitative and quantitative evaluations.

Effectively reconstructs complex geographic structures.

Synthesizes realistic objects across various classes.

Abstract

Aerial-to-ground image synthesis is an emerging and challenging problem that aims to synthesize a ground image from an aerial image. Due to the highly different layout and object representation between the aerial and ground images, existing approaches usually fail to transfer the components of the aerial scene into the ground scene. In this paper, we propose a novel framework to explore the challenges by imposing enhanced structural alignment and semantic awareness. We introduce a novel semantic-attentive feature transformation module that allows to reconstruct the complex geographic structures by aligning the aerial feature to the ground layout. Furthermore, we propose semantic-aware loss functions by leveraging a pre-trained segmentation network. The network is enforced to synthesize realistic objects across various classes by separately calculating losses for different classes and balancing them. Extensive experiments including comparisons with previous methods and ablation studies show the effectiveness of the proposed framework both qualitatively and quantitatively.