Local and Global Graph Modeling with Edge-weighted Graph Attention Network for Handwritten Mathematical Expression Recognition

TL;DR

This paper introduces an innovative graph-based end-to-end model with edge-weighted attention for handwritten mathematical expression recognition, effectively capturing local and global features to improve recognition accuracy.

Contribution

It presents a novel Edge-weighted Graph Attention Network and stroke-level graph modeling techniques for enhanced online handwritten mathematical expression recognition.

Findings

Superior performance in symbol detection and relation classification

Effective integration of local and global graph features

End-to-end recognition with improved accuracy

Abstract

In this paper, we present a novel approach to Handwritten Mathematical Expression Recognition (HMER) by leveraging graph-based modeling techniques. We introduce an End-to-end model with an Edge-weighted Graph Attention Mechanism (EGAT), designed to perform simultaneous node and edge classification. This model effectively integrates node and edge features, facilitating the prediction of symbol classes and their relationships within mathematical expressions. Additionally, we propose a stroke-level Graph Modeling method for both local (LGM) and global (GGM) information, which applies an end-to-end model to Online HMER tasks, transforming the recognition problem into node and edge classification tasks in graph structure. By capturing both local and global graph features, our method ensures comprehensive understanding of the expression structure. Through the combination of these components, our system demonstrates superior performance in symbol detection, relation classification, and expression-level recognition.