Illegible Text to Readable Text: An Image-to-Image Transformation using Conditional Sliced Wasserstein Adversarial Networks

TL;DR

This paper introduces HW2MP-GAN, a novel image-to-image translation model using conditional Sliced Wasserstein adversarial networks to convert illegible handwritten text images into clear machine-print images, improving recognition accuracy.

Contribution

The paper proposes a new GAN architecture incorporating SWD and U-Net for high-quality handwritten-to-machine-print image translation, outperforming existing models.

Findings

Outperforms baseline cGAN models by 30 in FHD

Achieves 0.6 lower Levenshtein distance

Improves word accuracy by 39% on IAM database

Abstract

Automatic text recognition from ancient handwritten record images is an important problem in the genealogy domain. However, critical challenges such as varying noise conditions, vanishing texts, and variations in handwriting make the recognition task difficult. We tackle this problem by developing a handwritten-to-machine-print conditional Generative Adversarial network (HW2MP-GAN) model that formulates handwritten recognition as a text-Image-to-text-Image translation problem where a given image, typically in an illegible form, is converted into another image, close to its machine-print form. The proposed model consists of three-components including a generator, and word-level and character-level discriminators. The model incorporates Sliced Wasserstein distance (SWD) and U-Net architectures in HW2MP-GAN for better quality image-to-image transformation. Our experiments reveal that HW2MP-GAN outperforms state-of-the-art baseline cGAN models by almost 30 in Frechet Handwritten Distance (FHD), 0.6 on average Levenshtein distance and 39% in word accuracy for image-to-image translation on IAM database. Further, HW2MP-GAN improves handwritten recognition word accuracy by 1.3% compared to baseline handwritten recognition models on the IAM database.