MALeR: Improving Compositional Fidelity in Layout-Guided Generation

TL;DR

MALeR enhances text-to-image generation by improving compositional fidelity, ensuring subjects stay within layouts, attributes are correctly bound, and complex scenes are accurately rendered.

Contribution

The paper introduces MALeR, a novel method that prevents subjects from outside-layout placement and reduces attribute leakage in compositional scene generation.

Findings

Superior compositional accuracy demonstrated

Improved attribute binding in complex scenes

Enhanced generation consistency

Abstract

Recent advances in text-to-image models have enabled a new era of creative and controllable image generation. However, generating compositional scenes with multiple subjects and attributes remains a significant challenge. To enhance user control over subject placement, several layout-guided methods have been proposed. However, these methods face numerous challenges, particularly in compositional scenes. Unintended subjects often appear outside the layouts, generated images can be out-of-distribution and contain unnatural artifacts, or attributes bleed across subjects, leading to incorrect visual outputs. In this work, we propose MALeR, a method that addresses each of these challenges. Given a text prompt and corresponding layouts, our method prevents subjects from appearing outside the given layouts while being in-distribution. Additionally, we propose a masked, attribute-aware binding mechanism that prevents attribute leakage, enabling accurate rendering of subjects with multiple attributes, even in complex compositional scenes. Qualitative and quantitative evaluation demonstrates that our method achieves superior performance in compositional accuracy, generation consistency, and attribute binding compared to previous work. MALeR is particularly adept at generating images of scenes with multiple subjects and multiple attributes per subject.