Content and Style Aware Generation of Text-line Images for Handwriting Recognition

TL;DR

This paper introduces a generative model for creating diverse, style-aware handwritten text-line images conditioned on content, which improves handwritten text recognition by augmenting training data with realistic synthetic samples.

Contribution

A novel style and content conditioned generative method for handwritten text-line images that enhances recognition performance and adapts to new styles using unlabeled data.

Findings

Generated data improves recognition accuracy.

Method outperforms existing synthetic data approaches.

Effective style adaptation with unlabeled images.

Abstract

Handwritten Text Recognition has achieved an impressive performance in public benchmarks. However, due to the high inter- and intra-class variability between handwriting styles, such recognizers need to be trained using huge volumes of manually labeled training data. To alleviate this labor-consuming problem, synthetic data produced with TrueType fonts has been often used in the training loop to gain volume and augment the handwriting style variability. However, there is a significant style bias between synthetic and real data which hinders the improvement of recognition performance. To deal with such limitations, we propose a generative method for handwritten text-line images, which is conditioned on both visual appearance and textual content. Our method is able to produce long text-line samples with diverse handwriting styles. Once properly trained, our method can also be adapted to new target data by only accessing unlabeled text-line images to mimic handwritten styles and produce images with any textual content. Extensive experiments have been done on making use of the generated samples to boost Handwritten Text Recognition performance. Both qualitative and quantitative results demonstrate that the proposed approach outperforms the current state of the art.