LDC Arabic Treebanks and Associated Corpora: Data Divisions Manual
Mona Diab, Nizar Habash, Owen Rambow, Ryan Roth

TL;DR
This paper presents a set of standardized rules for dividing Arabic treebank corpora into training, development, and test sets to ensure consistency across NLP research and applications.
Contribution
It introduces a systematic methodology for dividing Arabic treebank data, facilitating consistent and reproducible NLP experiments and model evaluations.
Findings
Established division rules for Arabic treebanks
Enhanced consistency in NLP research datasets
Supported reliable model training and evaluation
Abstract
The Linguistic Data Consortium (LDC) has developed hundreds of data corpora for natural language processing (NLP) research. Among these are a number of annotated treebank corpora for Arabic. Typically, these corpora consist of a single collection of annotated documents. NLP research, however, usually requires multiple data sets for the purposes of training models, developing techniques, and final evaluation. Therefore it becomes necessary to divide the corpora used into the required data sets (divisions). This document details a set of rules that have been defined to enable consistent divisions for old and new Arabic treebanks (ATB) and related corpora.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Mathematics, Computing, and Information Processing · Handwritten Text Recognition Techniques
