Reconciling Semantic Controllability and Diversity for Remote Sensing Image Synthesis with Hybrid Semantic Embedding

TL;DR

This paper introduces HySEGGAN, a novel hybrid semantic embedding approach that enhances controllability and diversity in remote sensing image synthesis, improving quality and state-of-the-art performance.

Contribution

The paper proposes a hybrid semantic embedding method and a semantic refinement network to better balance controllability and diversity in remote sensing image synthesis.

Findings

Achieves a better balance between semantic controllability and diversity.

Significantly improves the quality of synthesized images.

Outperforms existing methods as a data augmentation technique.

Abstract

Significant advancements have been made in semantic image synthesis in remote sensing. However, existing methods still face formidable challenges in balancing semantic controllability and diversity. In this paper, we present a Hybrid Semantic Embedding Guided Generative Adversarial Network (HySEGGAN) for controllable and efficient remote sensing image synthesis. Specifically, HySEGGAN leverages hierarchical information from a single source. Motivated by feature description, we propose a hybrid semantic Embedding method, that coordinates fine-grained local semantic layouts to characterize the geometric structure of remote sensing objects without extra information. Besides, a Semantic Refinement Network (SRN) is introduced, incorporating a novel loss function to ensure fine-grained semantic feedback. The proposed approach mitigates semantic confusion and prevents geometric pattern collapse. Experimental results indicate that the method strikes an excellent balance between semantic controllability and diversity. Furthermore, HySEGGAN significantly improves the quality of synthesized images and achieves state-of-the-art performance as a data augmentation technique across multiple datasets for downstream tasks.