Are conditional GANs explicitly conditional?

TL;DR

This paper reveals that standard cGANs are not explicitly conditional and introduces a new a contrario cGAN method that explicitly models conditionality, leading to improved performance across various image synthesis tasks.

Contribution

The paper's main novelty is the explicit modeling of conditionality in cGANs using a contrario loss and adverse examples, enhancing their effectiveness.

Findings

cGAN discriminator does not automatically learn conditionality

A contrario cGAN improves performance on multiple datasets

Significant reduction in FID, mIoU, RMSE log, and NDB metrics

Abstract

This paper proposes two important contributions for conditional Generative Adversarial Networks (cGANs) to improve the wide variety of applications that exploit this architecture. The first main contribution is an analysis of cGANs to show that they are not explicitly conditional. In particular, it will be shown that the discriminator and subsequently the cGAN does not automatically learn the conditionality between inputs. The second contribution is a new method, called a contrario cGAN, that explicitly models conditionality for both parts of the adversarial architecture via a novel a contrario loss that involves training the discriminator to learn unconditional (adverse) examples. This leads to a novel type of data augmentation approach for GANs (a contrario learning) which allows to restrict the search space of the generator to conditional outputs using adverse examples. Extensive experimentation is carried out to evaluate the conditionality of the discriminator by proposing a probability distribution analysis. Comparisons with the cGAN architecture for different applications show significant improvements in performance on well known datasets including, semantic image synthesis, image segmentation, monocular depth prediction and "single label"-to-image using different metrics including Fr\'echet Inception Distance (FID), mean Intersection over Union (mIoU), Root Mean Square Error log (RMSE log) and Number of statistically-Different Bins (NDB).