Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition

TL;DR

This paper investigates whether explicit n-gram language models still enhance deep neural network-based handwritten text recognition, finding that hybrid models outperform pure neural approaches across multiple datasets.

Contribution

It provides a comprehensive evaluation of n-gram models' impact on neural handwriting recognition architectures, demonstrating their continued relevance and optimal parameter settings.

Findings

N-gram models improve recognition accuracy across datasets

Hybrid models outperform pure neural models

Character language models yield the best results

Abstract

In recent advances in automatic text recognition (ATR), deep neural networks have demonstrated the ability to implicitly capture language statistics, potentially reducing the need for traditional language models. This study directly addresses whether explicit language models, specifically n-gram models, still contribute to the performance of state-of-the-art deep learning architectures in the field of handwriting recognition. We evaluate two prominent neural network architectures, PyLaia and DAN, with and without the integration of explicit n-gram language models. Our experiments on three datasets - IAM, RIMES, and NorHand v2 - at both line and page level, investigate optimal parameters for n-gram models, including their order, weight, smoothing methods and tokenization level. The results show that incorporating character or subword n-gram models significantly improves the performance of ATR models on all datasets, challenging the notion that deep learning models alone are sufficient for optimal performance. In particular, the combination of DAN with a character language model outperforms current benchmarks, confirming the value of hybrid approaches in modern document analysis systems.