MathReader : Text-to-Speech for Mathematical Documents

TL;DR

MathReader is a novel text-to-speech system that accurately reads mathematical documents by integrating OCR, a fine-tuned T5 model, and TTS, significantly reducing errors in mathematical content pronunciation.

Contribution

It introduces MathReader, a new approach combining OCR and language modeling to improve TTS accuracy for mathematical documents, outperforming existing solutions.

Findings

Lower Word Error Rate (WER) compared to Microsoft Edge and Adobe Acrobat.

Reduced WER from 0.510 to 0.281 over Microsoft Edge.

Reduced WER from 0.617 to 0.281 over Adobe Acrobat.

Abstract

TTS (Text-to-Speech) document reader from Microsoft, Adobe, Apple, and OpenAI have been serviced worldwide. They provide relatively good TTS results for general plain text, but sometimes skip contents or provide unsatisfactory results for mathematical expressions. This is because most modern academic papers are written in LaTeX, and when LaTeX formulas are compiled, they are rendered as distinctive text forms within the document. However, traditional TTS document readers output only the text as it is recognized, without considering the mathematical meaning of the formulas. To address this issue, we propose MathReader, which effectively integrates OCR, a fine-tuned T5 model, and TTS. MathReader demonstrated a lower Word Error Rate (WER) than existing TTS document readers, such as Microsoft Edge and Adobe Acrobat, when processing documents containing mathematical formulas. MathReader reduced the WER from 0.510 to 0.281 compared to Microsoft Edge, and from 0.617 to 0.281 compared to Adobe Acrobat. This will significantly contribute to alleviating the inconvenience faced by users who want to listen to documents, especially those who are visually impaired. The code is available at https://github.com/hyeonsieun/MathReader.