Layout-to-Image Translation with Double Pooling Generative Adversarial Networks

TL;DR

This paper introduces DPGAN, a novel GAN architecture with a double pooling module that enhances semantic dependency modeling for layout-to-image translation, resulting in more realistic and semantically consistent images.

Contribution

The paper proposes a new Double Pooling GAN with a Double Pooling Module that captures both short- and long-range semantic dependencies, improving image quality and semantic consistency.

Findings

Outperforms state-of-the-art methods on five datasets.

Effective semantic dependency modeling improves image realism.

Double pooling modules are versatile and can be integrated into other GANs.

Abstract

In this paper, we address the task of layout-to-image translation, which aims to translate an input semantic layout to a realistic image. One open challenge widely observed in existing methods is the lack of effective semantic constraints during the image translation process, leading to models that cannot preserve the semantic information and ignore the semantic dependencies within the same object. To address this issue, we propose a novel Double Pooing GAN (DPGAN) for generating photo-realistic and semantically-consistent results from the input layout. We also propose a novel Double Pooling Module (DPM), which consists of the Square-shape Pooling Module (SPM) and the Rectangle-shape Pooling Module (RPM). Specifically, SPM aims to capture short-range semantic dependencies of the input layout with different spatial scales, while RPM aims to capture long-range semantic dependencies from both horizontal and vertical directions. We then effectively fuse both outputs of SPM and RPM to further enlarge the receptive field of our generator. Extensive experiments on five popular datasets show that the proposed DPGAN achieves better results than state-of-the-art methods. Finally, both SPM and SPM are general and can be seamlessly integrated into any GAN-based architectures to strengthen the feature representation. The code is available at https://github.com/Ha0Tang/DPGAN.