LLM-supported document separation for printed reviews from zbMATH Open

TL;DR

This paper develops an optimized LLM-based pipeline for digitizing and segmenting mathematical documents from zbMATH Open, significantly improving machine processing and accessibility of scanned literature.

Contribution

It introduces a novel LLM fine-tuning and voting framework for document separation, outperforming traditional methods and processing over 810,000 documents into machine-readable format.

Findings

Mathpix identified as the best OCR tool for LaTeX conversion.

Achieved 97.5% accuracy in document text extraction.

Processed 810,977 documents into machine-readable text.

Abstract

This paper presents a specialized methodology for digitizing and segmenting mathematical documents from zbMATH Open, a comprehensive database of mathematical literature, to enhance machine processing capabilities. Currently, approximately 831,000 documents exist only in scanned volumes, which makes them not machine-processable. Furthermore, these scans often span multiple pages or share pages with other documents and incorporate diverse typesetting techniques, posing challenges for automated processing. To address these issues, we evaluate various Optical Character Recognition (OCR) tools and document separation techniques, proposing an optimized pipeline that outperforms existing approaches. Our study identifies Mathpix as the most effective OCR tool for LaTeX conversion, demonstrating superior performance based on BLEU and Edit Distance metrics. For document separation, we fine-tune generative Large Language Models (LLMs) and integrate them into a Majority Voting framework, achieving 97.5% accuracy when providing the text of the document. Additionally, our method identifies the start and end indexes for 90.6% of the test dataset, with an accuracy of 98.4% on applicable cases, resulting in an overall accuracy of 89.1% on the entire dataset. This approach surpasses traditional baselines, including regular expressions, ChatGPT-4o, and computer vision-based techniques. As a practical outcome, we process 810,977 mathematical documents into machine-readable text and extract precise document boundaries for 721,288 documents in LaTeX format. These contributions significantly improve accessibility for mathematical information retrieval systems, machine learning models, and related applications.