Towards Detecting, Recognizing, and Parsing the Address Information from Bangla Signboard: A Deep Learning-based Approach

TL;DR

This paper presents a comprehensive deep learning-based system for detecting, recognizing, correcting, and parsing Bangla address information from signboards, addressing the challenges of natural scene text extraction in low-resource languages.

Contribution

It introduces an end-to-end framework with novel datasets, comparative analysis of recognition models, and a transformer-based address text parser for Bangla signboards.

Findings

Created annotated and synthetic datasets for Bangla signboard text

Compared various CTC-based and Encoder-Decoder architectures for recognition

Developed a transformer-based address text correction and parsing system

Abstract

Retrieving textual information from natural scene images is an active research area in the field of computer vision with numerous practical applications. Detecting text regions and extracting text from signboards is a challenging problem due to special characteristics like reflecting lights, uneven illumination, or shadows found in real-life natural scene images. With the advent of deep learning-based methods, different sophisticated techniques have been proposed for text detection and text recognition from the natural scene. Though a significant amount of effort has been devoted to extracting natural scene text for resourceful languages like English, little has been done for low-resource languages like Bangla. In this research work, we have proposed an end-to-end system with deep learning-based models for efficiently detecting, recognizing, correcting, and parsing address information from Bangla signboards. We have created manually annotated datasets and synthetic datasets to train signboard detection, address text detection, address text recognition, address text correction, and address text parser models. We have conducted a comparative study among different CTC-based and Encoder-Decoder model architectures for Bangla address text recognition. Moreover, we have designed a novel address text correction model using a sequence-to-sequence transformer-based network to improve the performance of Bangla address text recognition model by post-correction. Finally, we have developed a Bangla address text parser using the state-of-the-art transformer-based pre-trained language model.