Deep Structured Generative Models

TL;DR

This paper introduces a deep structured generative model that enhances GANs with scene structure information using a stochastic and-or graph, enabling the generation of complex, structured images.

Contribution

It proposes a novel deep structured generative model that incorporates scene structure via sAOG to improve image generation quality and realism.

Findings

Successfully captures scene structures in generated images

Generates more realistic images of complex scenes

Effective in modeling intrinsic scene structures

Abstract

Deep generative models have shown promising results in generating realistic images, but it is still non-trivial to generate images with complicated structures. The main reason is that most of the current generative models fail to explore the structures in the images including spatial layout and semantic relations between objects. To address this issue, we propose a novel deep structured generative model which boosts generative adversarial networks (GANs) with the aid of structure information. In particular, the layout or structure of the scene is encoded by a stochastic and-or graph (sAOG), in which the terminal nodes represent single objects and edges represent relations between objects. With the sAOG appropriately harnessed, our model can successfully capture the intrinsic structure in the scenes and generate images of complicated scenes accordingly. Furthermore, a detection network is introduced to infer scene structures from a image. Experimental results demonstrate the effectiveness of our proposed method on both modeling the intrinsic structures, and generating realistic images.