Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems

TL;DR

This paper explores training strategies for multilingual OCR systems tailored for India, demonstrating that fine-tuning existing models yields better accuracy and efficiency, and introduces specialized models for government documents.

Contribution

It compares two training approaches for multilingual OCR, finds fine-tuning superior, and presents new models achieving state-of-the-art results and practical deployment metrics.

Findings

Fine-tuning yields better accuracy-latency trade-offs.

Chitrapathak-2 achieves 3-6x speedup and SOTA in Telugu.

Parichay extracts structured data with 89.8% accuracy.

Abstract

Designing Optical Character Recognition (OCR) systems for India requires balancing linguistic diversity, document heterogeneity, and deployment constraints. In this paper, we study two training strategies for building multilingual OCR systems with Vision-Language Models through the Chitrapathak series. We first follow a popular multimodal approach, pairing a generic vision encoder with a strong multilingual language model and training the system end-to-end for OCR. Alternatively, we explore fine-tuning an existing OCR model, despite not being trained for the target languages. Through extensive evaluation on multilingual Indic OCR benchmarks and deployment-oriented metrics, we find that the second strategy consistently achieves better accuracy-latency trade-offs. Chitrapathak-2 achieves 3-6x speedup over its predecessor with being state-of-the-art (SOTA) in Telugu (6.69 char ANLS) and second best in the rest. In addition, we present Parichay, an independent OCR model series designed specifically for 9 Indian government documents to extract structured key fields, achieving 89.8% Exact Match score with a faster inference. Together, these systems achieve SOTA performance and provide practical guidance for building production-scale OCR pipelines in the Indian context.