Benchmarking recognition results on word image datasets

TL;DR

This paper benchmarks the maximum recognition accuracy on various word image datasets using manual segmentation and commercial OCR, highlighting the impact of image degradations and annotation methods.

Contribution

It introduces a semi-automated pixel-level segmentation tool and provides a comprehensive benchmark across multiple datasets with different image qualities.

Findings

Recognition rates range from 79.6% to 89.3% across datasets.

Pixel-level annotation offers advantages for recognition accuracy.

Degradations and distortions significantly affect OCR performance.

Abstract

We have benchmarked the maximum obtainable recognition accuracy on various word image datasets using manual segmentation and a currently available commercial OCR. We have developed a Matlab program, with graphical user interface, for semi-automated pixel level segmentation of word images. We discuss the advantages of pixel level annotation. We have covered five databases adding up to over 3600 word images. These word images have been cropped from camera captured scene, born-digital and street view images. We recognize the segmented word image using the trial version of Nuance Omnipage OCR. We also discuss, how the degradations introduced during acquisition or inaccuracies introduced during creation of word images affect the recognition of the word present in the image. Word images for different kinds of degradations and correction for slant and curvy nature of words are also discussed. The word recognition rates obtained on ICDAR 2003, Sign evaluation, Street view, Born-digital and ICDAR 2011 datasets are 83.9%, 89.3%, 79.6%, 88.5% and 86.7% respectively.