HishabNet: Detection, Localization and Calculation of Handwritten Bengali Mathematical Expressions

TL;DR

This paper introduces HishabNet, a CNN-based object detection system utilizing YOLOv3 to recognize, locate, and evaluate handwritten Bengali mathematical expressions, achieving high accuracy with a new dataset.

Contribution

It presents a novel approach combining object detection and CNN classification for handwritten Bengali math expressions, along with a new dataset 'Hishab' for this task.

Findings

Achieved 98.6% mean average precision in detection

CNN backbone accuracy of 99.63% on NumtaDB

CNN backbone accuracy of 99.08% on CMATERdb

Abstract

Recently, recognition of handwritten Bengali letters and digits have captured a lot of attention among the researchers of the AI community. In this work, we propose a Convolutional Neural Network (CNN) based object detection model which can recognize and evaluate handwritten Bengali mathematical expressions. This method is able to detect multiple Bengali digits and operators and locate their positions in the image. With that information, it is able to construct numbers from series of digits and perform mathematical operations on them. For the object detection task, the state-of-the-art YOLOv3 algorithm was utilized. For training and evaluating the model, we have engineered a new dataset 'Hishab' which is the first Bengali handwritten digits dataset intended for object detection. The model achieved an overall validation mean average precision (mAP) of 98.6%. Also, the classification accuracy of the feature extractor backbone CNN used in our model was tested on two publicly available Bengali handwritten digits datasets: NumtaDB and CMATERdb. The backbone CNN achieved a test set accuracy of 99.6252% on NumtaDB and 99.0833% on CMATERdb.