DLT: Conditioned layout generation with Joint Discrete-Continuous Diffusion Layout Transformer

TL;DR

This paper introduces DLT, a transformer-based joint discrete-continuous diffusion model for conditioned layout generation, effectively handling mixed attribute types and outperforming existing models on multiple datasets.

Contribution

The paper presents a novel joint diffusion model with a flexible conditioning mechanism for layout generation involving mixed discrete and continuous attributes.

Findings

Outperforms state-of-the-art models on layout datasets

Effective conditioning on partial component attributes

Versatile joint diffusion process for mixed data types

Abstract

Generating visual layouts is an essential ingredient of graphic design. The ability to condition layout generation on a partial subset of component attributes is critical to real-world applications that involve user interaction. Recently, diffusion models have demonstrated high-quality generative performances in various domains. However, it is unclear how to apply diffusion models to the natural representation of layouts which consists of a mix of discrete (class) and continuous (location, size) attributes. To address the conditioning layout generation problem, we introduce DLT, a joint discrete-continuous diffusion model. DLT is a transformer-based model which has a flexible conditioning mechanism that allows for conditioning on any given subset of all the layout component classes, locations, and sizes. Our method outperforms state-of-the-art generative models on various layout generation datasets with respect to different metrics and conditioning settings. Additionally, we validate the effectiveness of our proposed conditioning mechanism and the joint continuous-diffusion process. This joint process can be incorporated into a wide range of mixed discrete-continuous generative tasks.