Robust Encoder-Decoder Learning Framework towards Offline Handwritten Mathematical Expression Recognition Based on Multi-Scale Deep Neural Network

TL;DR

This paper introduces a robust neural network framework combining multi-scale CNN and attention RNN to improve offline handwritten mathematical expression recognition, addressing symbol recognition and 2D structure understanding.

Contribution

It proposes a novel multi-scale deep neural network model that effectively recognizes complex handwritten mathematical expressions as LaTeX sequences.

Findings

Achieved a WER of 25.715%

Achieved an ExpRate of 28.216%

Demonstrated improved recognition accuracy over existing methods

Abstract

Offline handwritten mathematical expression recognition is a challenging task, because handwritten mathematical expressions mainly have two problems in the process of recognition. On one hand, it is how to correctly recognize different mathematical symbols. On the other hand, it is how to correctly recognize the two-dimensional structure existing in mathematical expressions. Inspired by recent work in deep learning, a new neural network model that combines a Multi-Scale convolutional neural network (CNN) with an Attention recurrent neural network (RNN) is proposed to identify two-dimensional handwritten mathematical expressions as one-dimensional LaTeX sequences. As a result, the model proposed in the present work has achieved a WER error of 25.715% and ExpRate of 28.216%.