Moyun: A Diffusion-Based Model for Style-Specific Chinese Calligraphy Generation

TL;DR

Moyun is a diffusion-based model that enables controllable, style-specific Chinese calligraphy generation by replacing the Unet with Vision Mamba and introducing a TripleLabel control mechanism, tested on a large dataset.

Contribution

The paper introduces Moyun, a novel diffusion model with a new control mechanism for generating Chinese calligraphy in specified styles, including unseen calligraphers.

Findings

Effective style control demonstrated on large dataset

Can generate calligraphy matching specific calligrapher styles

Outperforms existing style transfer methods in controllability

Abstract

Although Chinese calligraphy generation has achieved style transfer, generating calligraphy by specifying the calligrapher, font, and character style remains challenging. To address this, we propose a new Chinese calligraphy generation model 'Moyun' , which replaces the Unet in the Diffusion model with Vision Mamba and introduces the TripleLabel control mechanism to achieve controllable calligraphy generation. The model was tested on our large-scale dataset 'Mobao' of over 1.9 million images, and the results demonstrate that 'Moyun' can effectively control the generation process and produce calligraphy in the specified style. Even for calligraphy the calligrapher has not written, 'Moyun' can generate calligraphy that matches the style of the calligrapher.