FontAdapter: Instant Font Adaptation in Visual Text Generation

TL;DR

FontAdapter is a rapid, reference-based font adaptation framework for visual text generation that achieves high-quality, real-time font customization and editing without fine-tuning.

Contribution

It introduces a two-stage curriculum learning approach and synthetic datasets to enable instant adaptation to unseen fonts conditioned on glyph images.

Findings

Enables font customization in seconds without fine-tuning

Supports visual text editing and font style blending

Achieves high-quality, robust font transfer across languages

Abstract

Text-to-image diffusion models have significantly improved the seamless integration of visual text into diverse image contexts. Recent approaches further improve control over font styles through fine-tuning with predefined font dictionaries. However, adapting unseen fonts outside the preset is computationally expensive, often requiring tens of minutes, making real-time customization impractical. In this paper, we present FontAdapter, a framework that enables visual text generation in unseen fonts within seconds, conditioned on a reference glyph image. To this end, we find that direct training on font datasets fails to capture nuanced font attributes, limiting generalization to new glyphs. To overcome this, we propose a two-stage curriculum learning approach: FontAdapter first learns to extract font attributes from isolated glyphs and then integrates these styles into diverse natural backgrounds. To support this two-stage training scheme, we construct synthetic datasets tailored to each stage, leveraging large-scale online fonts effectively. Experiments demonstrate that FontAdapter enables high-quality, robust font customization across unseen fonts without additional fine-tuning during inference. Furthermore, it supports visual text editing, font style blending, and cross-lingual font transfer, positioning FontAdapter as a versatile framework for font customization tasks.