Multi-Agent Interactive Question Generation Framework for Long Document Understanding

TL;DR

This paper introduces an automated multi-agent framework for generating long-context questions in English and Arabic, improving LVLMs' understanding of extensive documents and addressing data scarcity in low-resource languages.

Contribution

It presents a fully automated, multi-agent interactive system for generating high-quality long-document questions, enhancing LVLM training for better long-context comprehension.

Findings

Generated questions are challenging for existing LVLMs.

The framework covers diverse domains and languages.

The approach reduces reliance on costly human annotation.

Abstract

Document Understanding (DU) in long-contextual scenarios with complex layouts remains a significant challenge in vision-language research. Although Large Vision-Language Models (LVLMs) excel at short-context DU tasks, their performance declines in long-context settings. A key limitation is the scarcity of fine-grained training data, particularly for low-resource languages such as Arabic. Existing state-of-the-art techniques rely heavily on human annotation, which is costly and inefficient. We propose a fully automated, multi-agent interactive framework to generate long-context questions efficiently. Our approach efficiently generates high-quality single- and multi-page questions for extensive English and Arabic documents, covering hundreds of pages across diverse domains. This facilitates the development of LVLMs with enhanced long-context understanding ability. Experimental results in this work have shown that our generated English and Arabic questions (\textbf{AraEngLongBench}) are quite challenging to major open- and close-source LVLMs. The code and data proposed in this work can be found in https://github.com/wangk0b/Multi_Agentic_QA_Long_Doc.git. Sample Question and Answer (QA) pairs and structured system prompts can be found in the Appendix.