Word Searching in Scene Image and Video Frame in Multi-Script Scenario using Dynamic Shape Coding

TL;DR

This paper introduces a novel multi-script word spotting framework using dynamic shape coding and script-wise keyword generation, significantly improving text retrieval accuracy in complex scene images and video frames across multiple scripts.

Contribution

The paper presents a new multi-script word spotting method with dynamic shape coding and on-the-fly script-specific keyword generation, addressing limitations of existing script-specific approaches.

Findings

Effective retrieval of keywords in scene images and videos across multiple scripts.

Improved text alignment and confusion reduction through dynamic shape coding.

Successful application to English, Bangla, and Devanagari scripts with promising results.

Abstract

Retrieval of text information from natural scene images and video frames is a challenging task due to its inherent problems like complex character shapes, low resolution, background noise, etc. Available OCR systems often fail to retrieve such information in scene/video frames. Keyword spotting, an alternative way to retrieve information, performs efficient text searching in such scenarios. However, current word spotting techniques in scene/video images are script-specific and they are mainly developed for Latin script. This paper presents a novel word spotting framework using dynamic shape coding for text retrieval in natural scene image and video frames. The framework is designed to search query keyword from multiple scripts with the help of on-the-fly script-wise keyword generation for the corresponding script. We have used a two-stage word spotting approach using Hidden Markov Model (HMM) to detect the translated keyword in a given text line by identifying the script of the line. A novel unsupervised dynamic shape coding based scheme has been used to group similar shape characters to avoid confusion and to improve text alignment. Next, the hypotheses locations are verified to improve retrieval performance. To evaluate the proposed system for searching keyword from natural scene image and video frames, we have considered two popular Indic scripts such as Bangla (Bengali) and Devanagari along with English. Inspired by the zone-wise recognition approach in Indic scripts[1], zone-wise text information has been used to improve the traditional word spotting performance in Indic scripts. For our experiment, a dataset consisting of images of different scenes and video frames of English, Bangla and Devanagari scripts were considered. The results obtained showed the effectiveness of our proposed word spotting approach.