Fused Text Recogniser and Deep Embeddings Improve Word Recognition and Retrieval

TL;DR

This paper proposes a method that combines OCR output with deep embeddings to enhance word recognition and retrieval accuracy, especially in challenging document collections like historic Hindi texts.

Contribution

It introduces a fusion approach of OCR and deep embeddings for improved word recognition and retrieval, validated on Hindi documents.

Findings

Word recognition rate improved by 1.4 points.

Retrieval mAP increased by 11.13 points.

Fusion method enhances robustness in difficult document collections.

Abstract

Recognition and retrieval of textual content from the large document collections have been a powerful use case for the document image analysis community. Often the word is the basic unit for recognition as well as retrieval. Systems that rely only on the text recogniser (OCR) output are not robust enough in many situations, especially when the word recognition rates are poor, as in the case of historic documents or digital libraries. An alternative has been word spotting based methods that retrieve/match words based on a holistic representation of the word. In this paper, we fuse the noisy output of text recogniser with a deep embeddings representation derived out of the entire word. We use average and max fusion for improving the ranked results in the case of retrieval. We validate our methods on a collection of Hindi documents. We improve word recognition rate by 1.4 and retrieval by 11.13 in the mAP.