GlyphBanana: Advancing Precise Text Rendering Through Agentic Workflows

TL;DR

GlyphBanana introduces an agentic workflow with auxiliary tools for precise text and formula rendering, significantly improving accuracy in complex character generation across various Text-to-Image models.

Contribution

It presents a training-free, adaptable method that enhances text rendering precision by injecting glyph templates into models' latent space and attention maps.

Findings

Outperforms existing baselines in rendering accuracy

Effective across multiple Text-to-Image models

Demonstrates superior precision in complex character generation

Abstract

Despite recent advances in generative models driving significant progress in text rendering, accurately generating complex text and mathematical formulas remains a formidable challenge. This difficulty primarily stems from the limited instruction-following capabilities of current models when encountering out-of-distribution prompts. To address this, we introduce GlyphBanana, alongside a corresponding benchmark specifically designed for rendering complex characters and formulas. GlyphBanana employs an agentic workflow that integrates auxiliary tools to inject glyph templates into both the latent space and attention maps, facilitating the iterative refinement of generated images. Notably, our training-free approach can be seamlessly applied to various Text-to-Image (T2I) models, achieving superior precision compared to existing baselines. Extensive experiments demonstrate the effectiveness of our proposed workflow. Associated code is publicly available at https://github.com/yuriYanZeXuan/GlyphBanana.