ScriptViT: Vision Transformer-Based Personalized Handwriting Generation

TL;DR

This paper introduces ScriptViT, a Vision Transformer-based framework for personalized handwriting generation that captures global stylistic patterns and enhances style fidelity using cross-attention and stroke-level analysis.

Contribution

The work presents a novel Vision Transformer style encoder and a cross-attention mechanism for improved personalized handwriting synthesis, addressing limitations of previous models.

Findings

Better capture of long-range stylistic patterns.

Enhanced style coherence in generated handwriting.

Improved interpretability through stroke attention analysis.

Abstract

Styled handwriting generation aims to synthesize handwritten text that looks both realistic and aligned with a specific writer's style. While recent approaches involving GAN, transformer and diffusion-based models have made progress, they often struggle to capture the full spectrum of writer-specific attributes, particularly global stylistic patterns that span long-range spatial dependencies. As a result, capturing subtle writer-specific traits such as consistent slant, curvature or stroke pressure, while keeping the generated text accurate is still an open problem. In this work, we present a unified framework designed to address these limitations. We introduce a Vision Transformer-based style encoder that learns global stylistic patterns from multiple reference images, allowing the model to better represent long-range structural characteristics of handwriting. We then integrate these style cues with the target text using a cross-attention mechanism, enabling the system to produce handwritten images that more faithfully reflect the intended style. To make the process more interpretable, we utilize Salient Stroke Attention Analysis (SSAA), which reveals the stroke-level features the model focuses on during style transfer. Together, these components lead to handwriting synthesis that is not only more stylistically coherent, but also easier to understand and analyze.