Layout-Corrector: Alleviating Layout Sticking Phenomenon in Discrete Diffusion Model

TL;DR

This paper introduces Layout-Corrector, a module that improves discrete diffusion models for layout generation by identifying and correcting inharmonious elements, thereby enhancing layout quality and robustness during generation.

Contribution

The paper proposes a novel layout-assessment module that works with existing DDMs to address the layout sticking phenomenon and improve generation performance.

Findings

Layout-Corrector effectively identifies erroneous tokens in layouts.

It enhances layout harmony and generation quality.

It mitigates performance drops during fast sampling.

Abstract

Layout generation is a task to synthesize a harmonious layout with elements characterized by attributes such as category, position, and size. Human designers experiment with the placement and modification of elements to create aesthetic layouts, however, we observed that current discrete diffusion models (DDMs) struggle to correct inharmonious layouts after they have been generated. In this paper, we first provide novel insights into layout sticking phenomenon in DDMs and then propose a simple yet effective layout-assessment module Layout-Corrector, which works in conjunction with existing DDMs to address the layout sticking problem. We present a learning-based module capable of identifying inharmonious elements within layouts, considering overall layout harmony characterized by complex composition. During the generation process, Layout-Corrector evaluates the correctness of each token in the generated layout, reinitializing those with low scores to the ungenerated state. The DDM then uses the high-scored tokens as clues to regenerate the harmonized tokens. Layout-Corrector, tested on common benchmarks, consistently boosts layout-generation performance when in conjunction with various state-of-the-art DDMs. Furthermore, our extensive analysis demonstrates that the Layout-Corrector (1) successfully identifies erroneous tokens, (2) facilitates control over the fidelity-diversity trade-off, and (3) significantly mitigates the performance drop associated with fast sampling.