Towards CGAN-based Satellite Image Synthesis with Partial Pixel-Wise Annotation

TL;DR

This paper investigates the challenge of using CGANs for satellite image synthesis with partially annotated data, proposing detail augmentation methods to improve image quality despite missing pixel-wise annotations.

Contribution

It introduces the first study on CGAN-based satellite image synthesis with incomplete annotations and proposes two novel detail augmentation techniques to address this issue.

Findings

Canny edge augmentation improves image synthesis quality.

Color assignment for missing annotations enhances CGAN performance.

Proposed methods effectively mitigate the impact of missing pixel annotations.

Abstract

Conditional Generative Adversarial Nets (CGANs) need a significantly huge dataset with a detailed pixel-wise annotation to generate high-quality images. Unfortunately, any amount of missing pixel annotations may significantly impact the result not only locally, but also in annotated areas. To the best of our knowledge, such a challenge has never been investigated in the broader field of GANs. In this paper, we take the first step in this direction to study the problem of CGAN-based satellite image synthesis given partially annotated images. We first define the problem of image synthesis using partially annotated data, and we discuss a scenario in which we face such a challenge. We then propose an effective solution called detail augmentation to address this problem. To do so, we tested two different approaches to augment details to compensate for missing pixel-wise annotations. In the first approach, we augmented the original images with their Canny edges to using the CGAN to compensate for the missing annotations. The second approach, however, attempted to assign a color to all pixels with missing annotation. Eventually, a different CGAN was trained to translate the new feature images into a final output.