DocTron-Formula: Generalized Formula Recognition in Complex and Structured Scenarios
Yufeng Zhong, Zhixiong Zeng, Lei Chen, Longrong Yang, Liming Zheng, Jing Huang, Siqi Yang, Lin Ma

TL;DR
DocTron-Formula introduces a unified vision-language framework for recognizing complex mathematical formulas, outperforming specialized models and enabling robust understanding of scientific documents across diverse styles and structures.
Contribution
It presents a generalized OCR approach for formulas using a unified model and introduces CSFormula, a large-scale dataset for complex scientific content recognition.
Findings
Achieves state-of-the-art accuracy on formula recognition tasks.
Outperforms specialized models in robustness and versatility.
Establishes a new paradigm for scientific document understanding.
Abstract
Optical Character Recognition (OCR) for mathematical formula is essential for the intelligent analysis of scientific literature. However, both task-specific and general vision-language models often struggle to handle the structural diversity, complexity, and real-world variability inherent in mathematical content. In this work, we present DocTron-Formula, a unified framework built upon general vision-language models, thereby eliminating the need for specialized architectures. Furthermore, we introduce CSFormula, a large-scale and challenging dataset that encompasses multidisciplinary and structurally complex formulas at the line, paragraph, and page levels. Through straightforward supervised fine-tuning, our approach achieves state-of-the-art performance across a variety of styles, scientific domains, and complex layouts. Experimental results demonstrate that our method not only…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHandwritten Text Recognition Techniques · Mathematics, Computing, and Information Processing · Image and Object Detection Techniques
