Uncovering the Handwritten Text in the Margins: End-to-end Handwritten Text Detection and Recognition
Liang Cheng, Jonas Frankem\"olle, Adam Axelsson, Ekta Vats

TL;DR
This paper introduces an end-to-end system for detecting and recognizing handwritten marginalia in historical documents, utilizing data augmentation and transfer learning to address limited training data, and demonstrating effectiveness on library collection data.
Contribution
It presents a novel integrated framework combining detection and recognition for handwritten marginalia, with techniques to mitigate data scarcity, and evaluates on real historical document data.
Findings
Effective detection and recognition of marginalia achieved
Data augmentation and transfer learning improve performance
Open-source code and models provided
Abstract
The pressing need for digitization of historical documents has led to a strong interest in designing computerised image processing methods for automatic handwritten text recognition. However, not much attention has been paid on studying the handwritten text written in the margins, i.e. marginalia, that also forms an important source of information. Nevertheless, training an accurate and robust recognition system for marginalia calls for data-efficient approaches due to the unavailability of sufficient amounts of annotated multi-writer texts. Therefore, this work presents an end-to-end framework for automatic detection and recognition of handwritten marginalia, and leverages data augmentation and transfer learning to overcome training data scarcity. The detection phase involves investigation of R-CNN and Faster R-CNN networks. The recognition phase includes an attention-based…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHandwritten Text Recognition Techniques · Image Processing and 3D Reconstruction
MethodsLib · 1x1 Convolution · *Communicated@Fast*How Do I Communicate to Expedia? · Average Pooling · Bottleneck Residual Block · Residual Connection · Batch Normalization · Residual Block · Global Average Pooling · Max Pooling
