Enhancing Energy Minimization Framework for Scene Text Recognition with Top-Down Cues

TL;DR

This paper enhances scene text recognition by integrating bottom-up character detections with top-down language cues within an energy minimization framework, achieving improved accuracy on multiple benchmarks.

Contribution

It introduces a novel energy minimization model that combines character detection scores with lexicon-based language priors for scene text recognition.

Findings

Outperforms comparable methods on multiple datasets

Integrating CNN features further improves accuracy

Rigorous analysis validates each step of the approach

Abstract

Recognizing scene text is a challenging problem, even more so than the recognition of scanned documents. This problem has gained significant attention from the computer vision community in recent years, and several methods based on energy minimization frameworks and deep learning approaches have been proposed. In this work, we focus on the energy minimization framework and propose a model that exploits both bottom-up and top-down cues for recognizing cropped words extracted from street images. The bottom-up cues are derived from individual character detections from an image. We build a conditional random field model on these detections to jointly model the strength of the detections and the interactions between them. These interactions are top-down cues obtained from a lexicon-based prior, i.e., language statistics. The optimal word represented by the text image is obtained by minimizing the energy function corresponding to the random field model. We evaluate our proposed algorithm extensively on a number of cropped scene text benchmark datasets, namely Street View Text, ICDAR 2003, 2011 and 2013 datasets, and IIIT 5K-word, and show better performance than comparable methods. We perform a rigorous analysis of all the steps in our approach and analyze the results. We also show that state-of-the-art convolutional neural network features can be integrated in our framework to further improve the recognition performance.