Pixel-wise Conditioning of Generative Adversarial Networks

TL;DR

This paper introduces a regularization method for conditioning GANs with pixel-wise constraints, especially effective when only a few pixels are known, balancing image quality and condition satisfaction.

Contribution

It proposes an explicit regularization term for pixel-wise conditioning in GANs, improving control over generated images with minimal pixel constraints.

Findings

Regularization improves control over pixel constraints.

Trade-off between image quality and condition satisfaction is adjustable.

Effective on MNIST and FashionMNIST datasets.

Abstract

Generative Adversarial Networks (GANs) have proven successful for unsupervised image generation. Several works extended GANs to image inpainting by conditioning the generation with parts of the image one wants to reconstruct. However, these methods have limitations in settings where only a small subset of the image pixels is known beforehand. In this paper, we study the effectiveness of conditioning GANs by adding an explicit regularization term to enforce pixel-wise conditions when very few pixel values are provided. In addition, we also investigate the influence of this regularization term on the quality of the generated images and the satisfaction of the conditions. Conducted experiments on MNIST and FashionMNIST show evidence that this regularization term allows for controlling the trade-off between quality of the generated images and constraint satisfaction.