Scan, Attend and Read: End-to-End Handwritten Paragraph Recognition with MDLSTM Attention

TL;DR

This paper introduces an end-to-end handwriting recognition model that uses MDLSTM attention to transcribe entire paragraphs without prior segmentation, marking a significant step forward in automatic handwritten text processing.

Contribution

The novel approach combines multi-dimensional LSTM with attention mechanisms for paragraph-level recognition, eliminating the need for line segmentation in handwriting recognition.

Findings

Successful end-to-end paragraph recognition on IAM Database

First to demonstrate multi-line handwriting recognition without segmentation

Encouraging results suggest feasibility of full paragraph transcription

Abstract

We present an attention-based model for end-to-end handwriting recognition. Our system does not require any segmentation of the input paragraph. The model is inspired by the differentiable attention models presented recently for speech recognition, image captioning or translation. The main difference is the covert and overt attention, implemented as a multi-dimensional LSTM network. Our principal contribution towards handwriting recognition lies in the automatic transcription without a prior segmentation into lines, which was crucial in previous approaches. To the best of our knowledge this is the first successful attempt of end-to-end multi-line handwriting recognition. We carried out experiments on the well-known IAM Database. The results are encouraging and bring hope to perform full paragraph transcription in the near future.