What is the Reward for Handwriting? -- Handwriting Generation by Imitation Learning

TL;DR

This paper introduces a reinforcement learning approach using generative adversarial imitation learning to generate handwriting by learning from examples, capturing the planning ability of human handwriting.

Contribution

It applies GAIL to handwriting generation, enabling the learning of reward functions directly from data, which improves the generation quality and understanding of handwriting behavior.

Findings

GAIL effectively captures handwriting trends

The learned reward function aligns with human handwriting patterns

The method outperforms traditional models in generating realistic handwriting

Abstract

Analyzing the handwriting generation process is an important issue and has been tackled by various generation models, such as kinematics based models and stochastic models. In this study, we use a reinforcement learning (RL) framework to realize handwriting generation with the careful future planning ability. In fact, the handwriting process of human beings is also supported by their future planning ability; for example, the ability is necessary to generate a closed trajectory like '0' because any shortsighted model, such as a Markovian model, cannot generate it. For the algorithm, we employ generative adversarial imitation learning (GAIL). Typical RL algorithms require the manual definition of the reward function, which is very crucial to control the generation process. In contrast, GAIL trains the reward function along with the other modules of the framework. In other words, through GAIL, we can understand the reward of the handwriting generation process from handwriting examples. Our experimental results qualitatively and quantitatively show that the learned reward catches the trends in handwriting generation and thus GAIL is well suited for the acquisition of handwriting behavior.