Structured Prediction using cGANs with Fusion Discriminator

TL;DR

This paper introduces a fusion discriminator framework for cGANs that effectively incorporates conditional information, improving structured prediction tasks like image synthesis, segmentation, and depth estimation without relying on specific potential functions.

Contribution

The paper presents a novel fusion discriminator that enforces higher-order consistency across various structured prediction tasks, offering a flexible alternative to CNN-CRF models.

Findings

Improved performance on multiple structured prediction tasks

Enforces higher-order consistency without specific potential functions

Flexible and conceptually simple framework

Abstract

We propose the fusion discriminator, a single unified framework for incorporating conditional information into a generative adversarial network (GAN) for a variety of distinct structured prediction tasks, including image synthesis, semantic segmentation, and depth estimation. Much like commonly used convolutional neural network -- conditional Markov random field (CNN-CRF) models, the proposed method is able to enforce higher-order consistency in the model, but without being limited to a very specific class of potentials. The method is conceptually simple and flexible, and our experimental results demonstrate improvement on several diverse structured prediction tasks.