A Layer-Based Sequential Framework for Scene Generation with GANs

TL;DR

This paper introduces a novel layer-based sequential GAN framework that generates complex scenes by separately creating backgrounds and foreground objects, resulting in more diverse and robust scene images.

Contribution

It proposes a new sequential scene generation method with explicit control over scene elements, improving diversity and handling of transformations and occlusions.

Findings

Produces more diverse scene images

Handles affine transformations better

Manages occlusion artifacts effectively

Abstract

The visual world we sense, interpret and interact everyday is a complex composition of interleaved physical entities. Therefore, it is a very challenging task to generate vivid scenes of similar complexity using computers. In this work, we present a scene generation framework based on Generative Adversarial Networks (GANs) to sequentially compose a scene, breaking down the underlying problem into smaller ones. Different than the existing approaches, our framework offers an explicit control over the elements of a scene through separate background and foreground generators. Starting with an initially generated background, foreground objects then populate the scene one-by-one in a sequential manner. Via quantitative and qualitative experiments on a subset of the MS-COCO dataset, we show that our proposed framework produces not only more diverse images but also copes better with affine transformations and occlusion artifacts of foreground objects than its counterparts.