Dolfin: Diffusion Layout Transformers without Autoencoder

TL;DR

Dolfin is a novel Transformer-based diffusion model for layout generation that improves modeling capability and reduces complexity, with applications in geometric structure modeling and enhanced performance on standard benchmarks.

Contribution

Introduces Dolfin, a diffusion layout transformer without autoencoder, featuring an autoregressive variant for capturing semantic object correlations, and demonstrates superior benchmark performance.

Findings

Significantly improves layout generation metrics

Effectively models geometric structures like line segments

Enhances transparency and interoperability

Abstract

In this paper, we introduce a novel generative model, Diffusion Layout Transformers without Autoencoder (Dolfin), which significantly improves the modeling capability with reduced complexity compared to existing methods. Dolfin employs a Transformer-based diffusion process to model layout generation. In addition to an efficient bi-directional (non-causal joint) sequence representation, we further propose an autoregressive diffusion model (Dolfin-AR) that is especially adept at capturing rich semantic correlations for the neighboring objects, such as alignment, size, and overlap. When evaluated against standard generative layout benchmarks, Dolfin notably improves performance across various metrics (fid, alignment, overlap, MaxIoU and DocSim scores), enhancing transparency and interoperability in the process. Moreover, Dolfin's applications extend beyond layout generation, making it suitable for modeling geometric structures, such as line segments. Our experiments present both qualitative and quantitative results to demonstrate the advantages of Dolfin.