Structure-Level Disentangled Diffusion for Few-Shot Chinese Font Generation

TL;DR

This paper introduces SLD-Font, a diffusion-based model for few-shot Chinese font generation that effectively disentangles content and style at the structure level, leading to higher style fidelity and good content accuracy.

Contribution

The paper proposes a novel structure-level disentangled diffusion model with separate content and style channels, and a parameter-efficient fine-tuning strategy for better style adaptation.

Findings

Achieves higher style fidelity than existing methods.

Maintains comparable content accuracy.

Introduces new metrics for content quality evaluation.

Abstract

Few-shot Chinese font generation aims to synthesize new characters in a target style using only a handful of reference images. Achieving accurate content rendering and faithful style transfer requires effective disentanglement between content and style. However, existing approaches achieve only feature-level disentanglement, allowing the generator to re-entangle these features, leading to content distortion and degraded style fidelity. We propose the Structure-Level Disentangled Diffusion Model (SLD-Font), which receives content and style information from two separate channels. SimSun-style images are used as content templates and concatenated with noisy latent features as the input. Style features extracted by a CLIP model from target-style images are integrated via cross-attention. Additionally, we train a Background Noise Removal module in the pixel space to remove background noise in complex stroke regions. Based on theoretical validation of disentanglement effectiveness, we introduce a parameter-efficient fine-tuning strategy that updates only the style-related modules. This allows the model to better adapt to new styles while avoiding overfitting to the reference images' content. We further introduce the Grey and OCR metrics to evaluate the content quality of generated characters. Experimental results show that SLD-Font achieves significantly higher style fidelity while maintaining comparable content accuracy to existing state-of-the-art methods.