ICAL: Implicit Character-Aided Learning for Enhanced Handwritten Mathematical Expression Recognition

TL;DR

The paper introduces ICAL, a novel method that leverages implicit character information to improve the accuracy of handwritten mathematical expression recognition, surpassing existing models on multiple datasets.

Contribution

The paper proposes the Implicit Character Construction Module and Fusion Module to effectively utilize implicit character information in recognition models, enhancing global understanding.

Findings

ICAL outperforms state-of-the-art models on CROHME datasets.

Achieves 69.06% accuracy on HME100k test set.

Improves expression recognition rates by up to 2.25%.

Abstract

Significant progress has been made in the field of handwritten mathematical expression recognition, while existing encoder-decoder methods are usually difficult to model global information in $LaTeX$. Therefore, this paper introduces a novel approach, Implicit Character-Aided Learning (ICAL), to mine the global expression information and enhance handwritten mathematical expression recognition. Specifically, we propose the Implicit Character Construction Module (ICCM) to predict implicit character sequences and use a Fusion Module to merge the outputs of the ICCM and the decoder, thereby producing corrected predictions. By modeling and utilizing implicit character information, ICAL achieves a more accurate and context-aware interpretation of handwritten mathematical expressions. Experimental results demonstrate that ICAL notably surpasses the state-of-the-art(SOTA) models, improving the expression recognition rate (ExpRate) by 2.25\%/1.81\%/1.39\% on the CROHME 2014/2016/2019 datasets respectively, and achieves a remarkable 69.06\% on the challenging HME100k test set. We make our code available on the GitHub: https://github.com/qingzhenduyu/ICAL