Dual Orthogonal Guidance for Robust Diffusion-based Handwritten Text Generation

TL;DR

This paper introduces Dual Orthogonal Guidance (DOG), a novel sampling strategy for diffusion-based handwritten text generation that enhances style variability and clarity by reducing artifacts and distortions.

Contribution

The paper proposes DOG, a new guidance method that improves diffusion-based handwritten text generation by stabilizing outputs and increasing diversity compared to standard guidance techniques.

Findings

DOG outperforms standard guidance in clarity and style diversity.

Experimental results show improved readability and style variability.

Effective on out-of-vocabulary words and complex styles.

Abstract

Diffusion-based Handwritten Text Generation (HTG) approaches achieve impressive results on frequent, in-vocabulary words observed at training time and on regular styles. However, they are prone to memorizing training samples and often struggle with style variability and generation clarity. In particular, standard diffusion models tend to produce artifacts or distortions that negatively affect the readability of the generated text, especially when the style is hard to produce. To tackle these issues, we propose a novel sampling guidance strategy, Dual Orthogonal Guidance (DOG), that leverages an orthogonal projection of a negatively perturbed prompt onto the original positive prompt. This approach helps steer the generation away from artifacts while maintaining the intended content, and encourages more diverse, yet plausible, outputs. Unlike standard Classifier-Free Guidance (CFG), which relies on unconditional predictions and produces noise at high guidance scales, DOG introduces a more stable, disentangled direction in the latent space. To control the strength of the guidance across the denoising process, we apply a triangular schedule: weak at the start and end of denoising, when the process is most sensitive, and strongest in the middle steps. Experimental results on the state-of-the-art DiffusionPen and One-DM demonstrate that DOG improves both content clarity and style variability, even for out-of-vocabulary words and challenging writing styles.