GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models

TL;DR

GlyphDraw2 is a novel framework that combines diffusion models and large language models to automatically generate complex, high-resolution posters with precise text rendering and controllable design features.

Contribution

It introduces a new automatic poster generation method leveraging LLMs and diffusion models, with a triple-cross attention mechanism for accurate text placement and style control.

Findings

Effective generation of posters with complex backgrounds

Supports multilingual text rendering in English and Chinese

Produces high-resolution posters exceeding 1024 pixels

Abstract

Posters play a crucial role in marketing and advertising by enhancing visual communication and brand visibility, making significant contributions to industrial design. With the latest advancements in controllable T2I diffusion models, increasing research has focused on rendering text within synthesized images. Despite improvements in text rendering accuracy, the field of automatic poster generation remains underexplored. In this paper, we propose an automatic poster generation framework with text rendering capabilities leveraging LLMs, utilizing a triple-cross attention mechanism based on alignment learning. This framework aims to create precise poster text within a detailed contextual background. Additionally, the framework supports controllable fonts, adjustable image resolution, and the rendering of posters with descriptions and text in both English and Chinese.Furthermore, we introduce a high-resolution font dataset and a poster dataset with resolutions exceeding 1024 pixels. Our approach leverages the SDXL architecture. Extensive experiments validate our method's capability in generating poster images with complex and contextually rich backgrounds.Codes is available at https://github.com/OPPO-Mente-Lab/GlyphDraw2.