Handwriting styles: benchmarks and evaluation metrics

TL;DR

This paper establishes benchmarks and evaluation metrics for handwriting style generation using deep learning, addressing the challenge of defining and assessing handwriting style quality.

Contribution

It introduces the first benchmarks and evaluation metrics for handwriting style generation, providing a foundation for future research in personalized handwriting systems.

Findings

Proposed baseline benchmarks for handwriting style evaluation

Developed relevant evaluation metrics for generative handwriting models

Analyzed challenges in evaluating handwriting style generation

Abstract

Evaluating the style of handwriting generation is a challenging problem, since it is not well defined. It is a key component in order to develop in developing systems with more personalized experiences with humans. In this paper, we propose baseline benchmarks, in order to set anchors to estimate the relative quality of different handwriting style methods. This will be done using deep learning techniques, which have shown remarkable results in different machine learning tasks, learning classification, regression, and most relevant to our work, generating temporal sequences. We discuss the challenges associated with evaluating our methods, which is related to evaluation of generative models in general. We then propose evaluation metrics, which we find relevant to this problem, and we discuss how we evaluate the evaluation metrics. In this study, we use IRON-OFF dataset. To the best of our knowledge, there is no work done before in generating handwriting (either in terms of methodology or the performance metrics), our in exploring styles using this dataset.