Learning symbol relation tree for online mathematical expression recognition

TL;DR

This paper introduces a novel approach for online handwritten mathematical expression recognition using a symbol relation tree, leveraging neural networks to improve accuracy and handle stroke order variations.

Contribution

It presents a new symbol relation tree-based recognition system combining a temporal classifier and tree connector, with a tree sorting method for stroke order variability.

Findings

Achieved 44.12% recognition rate on CROHME 2014

Achieved 41.76% recognition rate on CROHME 2016

System is competitive with existing methods

Abstract

This paper proposes a method for recognizing online handwritten mathematical expressions (OnHME) by building a symbol relation tree (SRT) directly from a sequence of strokes. A bidirectional recurrent neural network learns from multiple derived paths of SRT to predict both symbols and spatial relations between symbols using global context. The recognition system has two parts: a temporal classifier and a tree connector. The temporal classifier produces an SRT by recognizing an OnHME pattern. The tree connector splits the SRT into several sub-SRTs. The final SRT is formed by looking up the best combination among those sub-SRTs. Besides, we adopt a tree sorting method to deal with various stroke orders. Recognition experiments indicate that the proposed OnHME recognition system is competitive to other methods. The recognition system achieves 44.12% and 41.76% expression recognition rates on the Competition on Recognition of Online Handwritten Mathematical Expressions (CROHME) 2014 and 2016 testing sets.