UniGlyph: Unified Segmentation-Conditioned Diffusion for Precise Visual Text Synthesis

TL;DR

This paper introduces UniGlyph, a segmentation-guided diffusion framework that uses pixel-level text masks to improve the accuracy and style fidelity of visual text synthesis in images, surpassing previous methods.

Contribution

The paper presents a novel unified conditional input using pixel-level text masks and a diffusion model with adaptive glyph conditioning, achieving state-of-the-art results in text-to-image synthesis.

Findings

Outperforms prior methods on the AnyText benchmark

Excels at small text rendering and complex layout preservation

Introduces new benchmarks for layout and style evaluation

Abstract

Text-to-image generation has greatly advanced content creation, yet accurately rendering visual text remains a key challenge due to blurred glyphs, semantic drift, and limited style control. Existing methods often rely on pre-rendered glyph images as conditions, but these struggle to retain original font styles and color cues, necessitating complex multi-branch designs that increase model overhead and reduce flexibility. To address these issues, we propose a segmentation-guided framework that uses pixel-level visual text masks -- rich in glyph shape, color, and spatial detail -- as unified conditional inputs. Our method introduces two core components: (1) a fine-tuned bilingual segmentation model for precise text mask extraction, and (2) a streamlined diffusion model augmented with adaptive glyph conditioning and a region-specific loss to preserve textual fidelity in both content and style. Our approach achieves state-of-the-art performance on the AnyText benchmark, significantly surpassing prior methods in both Chinese and English settings. To enable more rigorous evaluation, we also introduce two new benchmarks: GlyphMM-benchmark for testing layout and glyph consistency in complex typesetting, and MiniText-benchmark for assessing generation quality in small-scale text regions. Experimental results show that our model outperforms existing methods by a large margin in both scenarios, particularly excelling at small text rendering and complex layout preservation, validating its strong generalization and deployment readiness.