Recognizing Handwritten Mathematical Expressions of Vertical Addition and Subtraction

TL;DR

This paper introduces a new dataset and method for recognizing handwritten vertical addition and subtraction expressions, demonstrating high accuracy with object detection algorithms and a novel transcription approach.

Contribution

The work presents the first dataset and recognition solution specifically for handwritten vertical elementary mathematical expressions.

Findings

High recognition accuracy achieved with object detection algorithms.

Effective mapping from detected symbols to LATEX expressions.

Dataset and code publicly available for further research.

Abstract

Handwritten Mathematical Expression Recognition (HMER) is a challenging task with many educational applications. Recent methods for HMER have been developed for complex mathematical expressions in standard horizontal format. However, solutions for elementary mathematical expression, such as vertical addition and subtraction, have not been explored in the literature. This work proposes a new handwritten elementary mathematical expression dataset composed of addition and subtraction expressions in a vertical format. We also extended the MNIST dataset to generate artificial images with this structure. Furthermore, we proposed a solution for offline HMER, able to recognize vertical addition and subtraction expressions. Our analysis evaluated the object detection algorithms YOLO v7, YOLO v8, YOLO-NAS, NanoDet and FCOS for identifying the mathematical symbols. We also proposed a transcription method to map the bounding boxes from the object detection stage to a mathematical expression in the LATEX markup sequence. Results show that our approach is efficient, achieving a high expression recognition rate. The code and dataset are available at https://github.com/Danielgol/HME-VAS