Decoupling Layout from Glyph in Online Chinese Handwriting Generation

TL;DR

This paper introduces a hierarchical approach to online Chinese handwriting generation by decoupling layout and glyph synthesis, enabling more realistic and style-consistent text line creation.

Contribution

It proposes a novel framework that separates layout generation from glyph synthesis, utilizing a diffusion-based model for stylized font creation in Chinese handwriting.

Findings

Generated text lines are structurally correct and style-consistent.

Method outperforms prior approaches in realism and style imitation.

Qualitative and quantitative results validate effectiveness.

Abstract

Text plays a crucial role in the transmission of human civilization, and teaching machines to generate online handwritten text in various styles presents an interesting and significant challenge. However, most prior work has concentrated on generating individual Chinese fonts, leaving {complete text line generation largely unexplored}. In this paper, we identify that text lines can naturally be divided into two components: layout and glyphs. Based on this division, we designed a text line layout generator coupled with a diffusion-based stylized font synthesizer to address this challenge hierarchically. More concretely, the layout generator performs in-context-like learning based on the text content and the provided style references to generate positions for each glyph autoregressively. Meanwhile, the font synthesizer which consists of a character embedding dictionary, a multi-scale calligraphy style encoder, and a 1D U-Net based diffusion denoiser will generate each font on its position while imitating the calligraphy style extracted from the given style references. Qualitative and quantitative experiments on the CASIA-OLHWDB demonstrate that our method is capable of generating structurally correct and indistinguishable imitation samples.