QARI-OCR: High-Fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation

TL;DR

Qari-OCR introduces a series of fine-tuned vision-language models that significantly improve Arabic OCR accuracy, especially for diacritically-rich texts, by leveraging synthetic datasets and multimodal adaptation.

Contribution

The paper presents Qari-OCR, a novel approach that adapts large multimodal models for high-fidelity Arabic text recognition, achieving state-of-the-art results and supporting diverse fonts, layouts, and handwriting.

Findings

QARI v0.2 achieves a WER of 0.160 and CER of 0.061.

Models outperform previous Arabic OCR methods on multiple metrics.

Open-source datasets and models facilitate further research.

Abstract

The inherent complexities of Arabic script; its cursive nature, diacritical marks (tashkeel), and varied typography, pose persistent challenges for Optical Character Recognition (OCR). We present Qari-OCR, a series of vision-language models derived from Qwen2-VL-2B-Instruct, progressively optimized for Arabic through iterative fine-tuning on specialized synthetic datasets. Our leading model, QARI v0.2, establishes a new open-source state-of-the-art with a Word Error Rate (WER) of 0.160, Character Error Rate (CER) of 0.061, and BLEU score of 0.737 on diacritically-rich texts. Qari-OCR demonstrates superior handling of tashkeel, diverse fonts, and document layouts, alongside impressive performance on low-resolution images. Further explorations (QARI v0.3) showcase strong potential for structural document understanding and handwritten text. This work delivers a marked improvement in Arabic OCR accuracy and efficiency, with all models and datasets released to foster further research.