IndicDLP: A Foundational Dataset for Multi-Lingual and Multi-Domain Document Layout Parsing

TL;DR

IndicDLP is a large, diverse dataset for multi-lingual and multi-domain document layout parsing, addressing limitations of existing datasets by including Indic languages, detailed labels, and broad domain coverage to improve document understanding models.

Contribution

The paper introduces IndicDLP, a comprehensive dataset for Indic language document layout analysis, and UED-mini for pretraining, enhancing model performance and generalization across languages and domains.

Findings

Fine-tuning models on IndicDLP improves accuracy.

Models trained on IndicDLP generalize well beyond Indic layouts.

IndicDLP bridges scale, diversity, and annotation gaps in document datasets.

Abstract

Document layout analysis is essential for downstream tasks such as information retrieval, extraction, OCR, and digitization. However, existing large-scale datasets like PubLayNet and DocBank lack fine-grained region labels and multilingual diversity, making them insufficient for representing complex document layouts. In contrast, human-annotated datasets such as M6Doc and D4LA offer richer labels and greater domain diversity, but are too small to train robust models and lack adequate multilingual coverage. This gap is especially pronounced for Indic documents, which encompass diverse scripts yet remain underrepresented in current datasets, further limiting progress in this space. To address these shortcomings, we introduce IndicDLP, a large-scale foundational document layout dataset spanning 11 representative Indic languages alongside English and 12 common document domains. Additionally, we curate UED-mini, a dataset derived from DocLayNet and M6Doc, to enhance pretraining and provide a solid foundation for Indic layout models. Our experiments demonstrate that fine-tuning existing English models on IndicDLP significantly boosts performance, validating its effectiveness. Moreover, models trained on IndicDLP generalize well beyond Indic layouts, making it a valuable resource for document digitization. This work bridges gaps in scale, diversity, and annotation granularity, driving inclusive and efficient document understanding.