HWD: A Novel Evaluation Score for Styled Handwritten Text Generation

TL;DR

This paper introduces HWD, a new evaluation score for styled handwritten text generation that measures handwriting style similarity using a specialized neural network and perceptual distance, aiding progress in this research area.

Contribution

The paper proposes the Handwriting Distance (HWD), a novel evaluation metric specifically designed for assessing styled handwritten text generation quality.

Findings

HWD effectively evaluates handwriting style similarity.

Experimental results show HWD correlates well with human judgment.

HWD outperforms existing evaluation metrics for HTG.

Abstract

Styled Handwritten Text Generation (Styled HTG) is an important task in document analysis, aiming to generate text images with the handwriting of given reference images. In recent years, there has been significant progress in the development of deep learning models for tackling this task. Being able to measure the performance of HTG models via a meaningful and representative criterion is key for fostering the development of this research topic. However, despite the current adoption of scores for natural image generation evaluation, assessing the quality of generated handwriting remains challenging. In light of this, we devise the Handwriting Distance (HWD), tailored for HTG evaluation. In particular, it works in the feature space of a network specifically trained to extract handwriting style features from the variable-lenght input images and exploits a perceptual distance to compare the subtle geometric features of handwriting. Through extensive experimental evaluation on different word-level and line-level datasets of handwritten text images, we demonstrate the suitability of the proposed HWD as a score for Styled HTG. The pretrained model used as backbone will be released to ease the adoption of the score, aiming to provide a valuable tool for evaluating HTG models and thus contributing to advancing this important research area.