TextAlign: Preference Alignment for Text Rendering with Hierarchical Rewards

TL;DR

TextAlign introduces a hierarchical reward-based framework to improve text rendering accuracy in image generation models without altering their architecture.

Contribution

It proposes a scalable, non-invasive preference-alignment method using hierarchical rewards to enhance text rendering in large models.

Findings

Consistent improvements in OCR-based text accuracy on benchmark datasets.

Outperforms existing baselines like SD3.5, Qwen-Image, and TextDiffuser.

Enhances text rendering without degrading overall image quality.

Abstract

Faithful text rendering remains a persistent weakness of large text-to-image generative models, as it requires both semantic instruction following and fine-grained glyph-level structure. Prior methods often improve this ability through architecture-specific modules or encoder modifications, which complicate deployment across foundation models. We study text rendering as a post-training preference-alignment problem and propose TextAlign, a non-invasive framework that keeps the generator architecture unchanged. The key component is a hierarchical vision-language model (VLM)-based reward that decomposes rendering errors into global, word, and glyph levels, then converts binary defect judgments into a scalar preference signal. The resulting signal supports both Group Relative Policy Optimization (GRPO) and Direct Preference Optimization (DPO). Experiments on FLUX.1-dev and Z-Image-Turbo show consistent gains in OCR-based text accuracy without degrading general generation quality. Compared with strong foundation and text-rendering baselines, including SD3.5, Qwen-Image, AnyText, and TextDiffuser, these results indicate that reward design offers a scalable alternative to model redesign for improving text rendering.