A Comprehensive Comparison of End-to-End Approaches for Handwritten Digit String Recognition

TL;DR

This paper compares end-to-end methods for handwritten digit string recognition, highlighting the effectiveness of object detection models like Yolo over traditional segmentation-based approaches across multiple datasets.

Contribution

It provides a comprehensive evaluation and critical analysis of object detection and sequence-to-sequence methods for HDSR on five benchmark datasets.

Findings

Yolo outperforms segmentation-free models with shorter pipelines

Yolo achieves 97% recognition on NIST-SD19

Yolo achieves 84% recognition on CVL dataset

Abstract

Over the last decades, most approaches proposed for handwritten digit string recognition (HDSR) have resorted to digit segmentation, which is dominated by heuristics, thereby imposing substantial constraints on the final performance. Few of them have been based on segmentation-free strategies where each pixel column has a potential cut location. Recently, segmentation-free strategies has added another perspective to the problem, leading to promising results. However, these strategies still show some limitations when dealing with a large number of touching digits. To bridge the resulting gap, in this paper, we hypothesize that a string of digits can be approached as a sequence of objects. We thus evaluate different end-to-end approaches to solve the HDSR problem, particularly in two verticals: those based on object-detection (e.g., Yolo and RetinaNet) and those based on sequence-to-sequence representation (CRNN). The main contribution of this work lies in its provision of a comprehensive comparison with a critical analysis of the above mentioned strategies on five benchmarks commonly used to assess HDSR, including the challenging Touching Pair dataset, NIST SD19, and two real-world datasets (CAR and CVL) proposed for the ICFHR 2014 competition on HDSR. Our results show that the Yolo model compares favorably against segmentation-free models with the advantage of having a shorter pipeline that minimizes the presence of heuristics-based models. It achieved a 97%, 96%, and 84% recognition rate on the NIST-SD19, CAR, and CVL datasets, respectively.