Development of a New Image-to-text Conversion System for Pashto, Farsi and Traditional Chinese

TL;DR

This paper presents Worldly OCR, a deep learning-based system for accurate image-to-text conversion across multiple languages and scripts, including cursive and non-cursive writing systems, targeting large-scale digital document processing.

Contribution

It introduces a novel OCR approach tailored for diverse scripts like Pashto, Farsi, and Traditional Chinese, handling large character sets and cursive writing.

Findings

Achieved high accuracy in cursive scripts Pashto and Farsi.

Developed methods for Traditional Chinese with 65,000 characters.

Scalable system designed for over a billion pages.

Abstract

We report upon the results of a research and prototype building project \emph{Worldly~OCR} dedicated to developing new, more accurate image-to-text conversion software for several languages and writing systems. These include the cursive scripts Farsi and Pashto, and Latin cursive scripts. We also describe approaches geared towards Traditional Chinese, which is non-cursive, but features an extremely large character set of 65,000 characters. Our methodology is based on Machine Learning, especially Deep Learning, and Data Science, and is directed towards vast quantities of original documents, exceeding a billion pages. The target audience of this paper is a general audience with interest in Digital Humanities or in retrieval of accurate full-text and metadata from digital images.