TextPixs: Glyph-Conditioned Diffusion with Character-Aware Attention and OCR-Guided Supervision

TL;DR

This paper introduces GCDA, a diffusion-based model that generates readable, correctly spelled text in images by combining glyph-aware encoding, character-specific attention, and OCR-guided fine-tuning, achieving state-of-the-art results.

Contribution

The paper proposes a novel glyph-conditioned diffusion framework with character-aware attention and OCR supervision, improving text readability and accuracy in generated images.

Findings

Achieves lower Character Error Rate (0.08) compared to previous models (0.21).

Outperforms in human perception and text rendering metrics.

Maintains high image quality with FID of 14.3.

Abstract

The modern text-to-image diffusion models boom has opened a new era in digital content production as it has proven the previously unseen ability to produce photorealistic and stylistically diverse imagery based on the semantics of natural-language descriptions. However, the consistent disadvantage of these models is that they cannot generate readable, meaningful, and correctly spelled text in generated images, which significantly limits the use of practical purposes like advertising, learning, and creative design. This paper introduces a new framework, namely Glyph-Conditioned Diffusion with Character-Aware Attention (GCDA), using which a typical diffusion backbone is extended by three well-designed modules. To begin with, the model has a dual-stream text encoder that encodes both semantic contextual information and explicit glyph representations, resulting in a character-aware representation of the input text that is rich in nature. Second, an attention mechanism that is aware of the character is proposed with a new attention segregation loss that aims to limit the attention distribution of each character independently in order to avoid distortion artifacts. Lastly, GCDA has an OCR-in-the-loop fine-tuning phase, where a full text perceptual loss, directly optimises models to be legible and accurately spell. Large scale experiments to benchmark datasets, such as MARIO-10M and T2I-CompBench, reveal that GCDA sets a new state-of-the-art on all metrics, with better character based metrics on text rendering (Character Error Rate: 0.08 vs 0.21 for the previous best; Word Error Rate: 0.15 vs 0.25), human perception, and comparable image synthesis quality on high-fidelity (FID: 14.3).