Handwriting recognition using Cohort of LSTM and lexicon verification with extremely large lexicon

TL;DR

This paper introduces a novel handwriting recognition approach combining a cohort of LSTM networks with lexicon verification, enabling effective recognition with extremely large lexicons of millions of words.

Contribution

It proposes a cascade architecture with a cohort of LSTM networks and a lexicon verification process, surpassing existing methods on large lexicons and improving recognition performance.

Findings

Achieved state-of-the-art results on Rimes and IAM datasets.

Effectively handled a 3-million-word lexicon with fast decision times.

Demonstrated the effectiveness of cohort-based ensemble learning.

Abstract

State-of-the-art methods for handwriting recognition are based on Long Short Term Memory (LSTM) recurrent neural networks (RNN), which now provides very impressive character recognition performance. The character recognition is generally coupled with a lexicon driven decoding process which integrates dictionaries. Unfortunately these dictionaries are limited to hundred of thousands words for the best systems, which prevent from having a good language coverage, and therefore limit the global recognition performance. In this article, we propose an alternative to the lexicon driven decoding process based on a lexicon verification process, coupled with an original cascade architecture. The cascade is made of a large number of complementary networks extracted from a single training (called cohort), making the learning process very light. The proposed method achieves new state-of-the art word recognition performance on the Rimes and IAM databases. Dealing with gigantic lexicon of 3 millions words, the methods also demonstrates interesting performance with a fast decision stage.