Date-Field Retrieval in Scene Image and Video Frames using Text Enhancement and Shape Coding

TL;DR

This paper introduces a novel date spotting system for scene images and videos that enhances text recognition through image processing, shape coding, and HMM-based detection, enabling effective date retrieval despite complex backgrounds.

Contribution

It proposes a new line-based date spotting method using shape coding and feature fusion with HMM, improving date detection in challenging scene images and videos.

Findings

Effective date spotting on 1648 text lines.

Improved retrieval accuracy with shape coding scheme.

Enhanced text features using wavelet and PHOG techniques.

Abstract

Text recognition in scene image and video frames is difficult because of low resolution, blur, background noise, etc. Since traditional OCRs do not perform well in such images, information retrieval using keywords could be an alternative way to index/retrieve such text information. Date is a useful piece of information which has various applications including date-wise videos/scene searching, indexing or retrieval. This paper presents a date spotting based information retrieval system for natural scene image and video frames where text appears with complex backgrounds. We propose a line based date spotting approach using Hidden Markov Model (HMM) which is used to detect the date information in a given text. Different date models are searched from a line without segmenting characters or words. Given a text line image in RGB, we apply an efficient gray image conversion to enhance the text information. Wavelet decomposition and gradient sub-bands are used to enhance text information in gray scale. Next, Pyramid Histogram of Oriented Gradient (PHOG) feature has been extracted from gray image and binary images for date-spotting framework. Binary and gray image features are combined by MLP based Tandem approach. Finally, to boost the performance further, a shape coding based scheme is used to combine the similar shape characters in same class during word spotting. For our experiment, three different date models have been constructed to search similar date information having numeric dates that contains numeral values and punctuations and semi-numeric that contains dates with numerals along with months in scene/video text. We have tested our system on 1648 text lines and the results show the effectiveness of our proposed date spotting approach.