Complex Mathematical Expression Recognition: Benchmark, Large-Scale Dataset and Strong Baseline

TL;DR

This paper introduces a comprehensive benchmark, large-scale datasets, and a specialized model for recognizing complex mathematical expressions, significantly advancing the robustness and accuracy of mathematical expression recognition systems.

Contribution

The paper presents CMER-Bench, large datasets CMER-17M and CMER-3M, a novel expression tokenizer, Structured Mathematical Language, and the CMERNet model for improved complex mathematical expression recognition.

Findings

CMERNet outperforms existing models on CMER-Bench.

Current models struggle with complex expressions, especially on existing datasets.

Large-scale datasets and new representations improve recognition accuracy.

Abstract

Mathematical Expression Recognition (MER) has made significant progress in recognizing simple expressions, but the robust recognition of complex mathematical expressions with many tokens and multiple lines remains a formidable challenge. In this paper, we first introduce CMER-Bench, a carefully constructed benchmark that categorizes expressions into three difficulty levels: easy, moderate, and complex. Leveraging CMER-Bench, we conduct a comprehensive evaluation of existing MER models and general-purpose multimodal large language models (MLLMs). The results reveal that while current methods perform well on easy and moderate expressions, their performance degrades significantly when handling complex mathematical expressions, mainly because existing public training datasets are primarily composed of simple samples. In response, we propose MER-17M and CMER-3M that are large-scale datasets emphasizing the recognition of complex mathematical expressions. The datasets provide rich and diverse samples to support the development of accurate and robust complex MER models. Furthermore, to address the challenges posed by the complicated spatial layout of complex expressions, we introduce a novel expression tokenizer, and a new representation called Structured Mathematical Language, which explicitly models the hierarchical and spatial structure of expressions beyond LaTeX format. Based on these, we propose a specialized model named CMERNet, built upon an encoder-decoder architecture and trained on CMER-3M. Experimental results show that CMERNet, with only 125 million parameters, significantly outperforms existing MER models and MLLMs on CMER-Bench.