FontUse: A Data-Centric Approach to Style- and Use-Case-Conditioned In-Image Typography

TL;DR

This paper introduces FontUse, a data-centric method that enhances text-to-image models to better control typography by training on a large, annotated dataset focused on style and use-case conditions, improving prompt adherence.

Contribution

The paper presents a novel annotation pipeline and dataset, FontUse, enabling fine-tuning of image generation models for improved typography control without architectural changes.

Findings

Models trained with FontUse annotations better match typographic prompts.

The Long-CLIP metric effectively measures typography-prompt alignment.

FontUse improves consistency of generated text with specified styles and use cases.

Abstract

Recent text-to-image models can generate high-quality images from natural-language prompts, yet controlling typography remains challenging: requested typographic appearance is often ignored or only weakly followed. We address this limitation with a data-centric approach that trains image generation models using targeted supervision derived from a structured annotation pipeline specialized for typography. Our pipeline constructs a large-scale typography-focused dataset, FontUse, consisting of about 70K images annotated with user-friendly prompts, text-region locations, and OCR-recognized strings. The annotations are automatically produced using segmentation models and multimodal large language models (MLLMs). The prompts explicitly combine font styles (e.g., serif, script, elegant) and use cases (e.g., wedding invitations, coffee-shop menus), enabling intuitive specification even for novice users. Fine-tuning existing generators with these annotations allows them to consistently interpret style and use-case conditions as textual prompts without architectural modification. For evaluation, we introduce a Long-CLIP-based metric that measures alignment between generated typography and requested attributes. Experiments across diverse prompts and layouts show that models trained with our pipeline produce text renderings more consistent with prompts than competitive baselines. The source code for our annotation pipeline is available at https://github.com/xiaxinz/FontUSE.