UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition

TL;DR

This paper presents UniMER, a large-scale dataset and a specialized neural network, UniMERNet, for recognizing complex mathematical expressions in real-world scenarios, significantly improving accuracy and robustness.

Contribution

The paper introduces the first large-scale, diverse dataset for real-world mathematical expression recognition and proposes a tailored neural network architecture, UniMERNet, for improved performance.

Findings

Training on UniMER-1M improves model generalization.

UniMERNet achieves higher accuracy than previous models.

The dataset enables more comprehensive evaluation of MER systems.

Abstract

The paper introduces the UniMER dataset, marking the first study on Mathematical Expression Recognition (MER) targeting complex real-world scenarios. The UniMER dataset includes a large-scale training set, UniMER-1M, which offers unprecedented scale and diversity with one million training instances to train high-quality, robust models. Additionally, UniMER features a meticulously designed, diverse test set, UniMER-Test, which covers a variety of formula distributions found in real-world scenarios, providing a more comprehensive and fair evaluation. To better utilize the UniMER dataset, the paper proposes a Universal Mathematical Expression Recognition Network (UniMERNet), tailored to the characteristics of formula recognition. UniMERNet consists of a carefully designed encoder that incorporates detail-aware and local context features, and an optimized decoder for accelerated performance. Extensive experiments conducted using the UniMER-1M dataset and UniMERNet demonstrate that training on the large-scale UniMER-1M dataset can produce a more generalizable formula recognition model, significantly outperforming all previous datasets. Furthermore, the introduction of UniMERNet enhances the model's performance in formula recognition, achieving higher accuracy and speeds. All data, models, and code are available at https://github.com/opendatalab/UniMERNet.