The Learnable Typewriter: A Generative Approach to Text Analysis

TL;DR

This paper introduces a novel generative, unsupervised method for character analysis and recognition in text lines, capable of learning from limited data and applying to diverse historical and cipher texts.

Contribution

It adapts and evaluates deep unsupervised multi-object segmentation for real text images, enabling weakly supervised learning and new applications in paleography and cipher analysis.

Findings

Successfully applied to real text images and diverse datasets

Demonstrated effectiveness in paleography and cipher analysis

Achieved quantitative evaluation of unsupervised segmentation methods

Abstract

We present a generative document-specific approach to character analysis and recognition in text lines. Our main idea is to build on unsupervised multi-object segmentation methods and in particular those that reconstruct images based on a limited amount of visual elements, called sprites. Taking as input a set of text lines with similar font or handwriting, our approach can learn a large number of different characters and leverage line-level annotations when available. Our contribution is twofold. First, we provide the first adaptation and evaluation of a deep unsupervised multi-object segmentation approach for text line analysis. Since these methods have mainly been evaluated on synthetic data in a completely unsupervised setting, demonstrating that they can be adapted and quantitatively evaluated on real images of text and that they can be trained using weak supervision are significant progresses. Second, we show the potential of our method for new applications, more specifically in the field of paleography, which studies the history and variations of handwriting, and for cipher analysis. We demonstrate our approach on three very different datasets: a printed volume of the Google1000 dataset, the Copiale cipher and historical handwritten charters from the 12th and early 13th century.