Synthetic Document Question Answering in Hungarian

TL;DR

This paper introduces Hungarian document VQA datasets, HuDocVQA and HuDocVQA-manual, along with HuCCPDF for OCR training, to improve multilingual document question answering in low-resource languages.

Contribution

The paper presents new synthetic and manual Hungarian document VQA datasets and a large OCR dataset, enabling better model training and evaluation in low-resource language settings.

Findings

Finetuning on proposed datasets improves VQA accuracy by +7.2%.

Datasets are publicly released for further research.

Methodology for dataset quality filtering and deduplication.

Abstract

Modern VLMs have achieved near-saturation accuracy in English document visual question-answering (VQA). However, this task remains challenging in lower resource languages due to a dearth of suitable training and evaluation data. In this paper we present scalable methods for curating such datasets by focusing on Hungarian, approximately the 17th highest resource language on the internet. Specifically, we present HuDocVQA and HuDocVQA-manual, document VQA datasets that modern VLMs significantly underperform on compared to English DocVQA. HuDocVQA-manual is a small manually curated dataset based on Hungarian documents from Common Crawl, while HuDocVQA is a larger synthetically generated VQA data set from the same source. We apply multiple rounds of quality filtering and deduplication to HuDocVQA in order to match human-level quality in this dataset. We also present HuCCPDF, a dataset of 117k pages from Hungarian Common Crawl PDFs along with their transcriptions, which can be used for training a model for Hungarian OCR. To validate the quality of our datasets, we show how finetuning on a mixture of these datasets can improve accuracy on HuDocVQA for Llama 3.2 11B Instruct by +7.2%. Our datasets and code will be released to the public to foster further research in multilingual DocVQA.