Handwriting Transformers

TL;DR

This paper introduces HWT, a transformer-based model for generating styled handwritten text images that captures style-content relationships and outperforms existing methods in realism and versatility.

Contribution

It is the first to apply transformer architecture to styled handwritten text generation, enabling better style-content entanglement and generalization in few-shot scenarios.

Findings

HWT outperforms state-of-the-art methods in quality and realism.

It effectively handles arbitrary text lengths and unseen styles.

Generates realistic handwritten text in diverse styles.

Abstract

We propose a novel transformer-based styled handwritten text image generation approach, HWT, that strives to learn both style-content entanglement as well as global and local writing style patterns. The proposed HWT captures the long and short range relationships within the style examples through a self-attention mechanism, thereby encoding both global and local style patterns. Further, the proposed transformer-based HWT comprises an encoder-decoder attention that enables style-content entanglement by gathering the style representation of each query character. To the best of our knowledge, we are the first to introduce a transformer-based generative network for styled handwritten text generation. Our proposed HWT generates realistic styled handwritten text images and significantly outperforms the state-of-the-art demonstrated through extensive qualitative, quantitative and human-based evaluations. The proposed HWT can handle arbitrary length of text and any desired writing style in a few-shot setting. Further, our HWT generalizes well to the challenging scenario where both words and writing style are unseen during training, generating realistic styled handwritten text images.