Synthetic Context Generation for Question Generation

TL;DR

This paper explores using synthetic contexts generated by large language models to train question generation systems, demonstrating that synthetic contexts can be as effective as real ones and enabling better performance with smaller models.

Contribution

It introduces a method for training question generation models with synthetic contexts, showing their effectiveness and potential to reduce reliance on domain-specific datasets.

Findings

Synthetic contexts are crucial for effective question generation.

Smaller models fine-tuned on synthetic contexts outperform larger models using prompting.

Synthetic and real contexts yield comparable question generation performance.

Abstract

Despite rapid advancements in large language models (LLMs), QG remains a challenging problem due to its complicated process, open-ended nature, and the diverse settings in which question generation occurs. A common approach to address these challenges involves fine-tuning smaller, custom models using datasets containing background context, question, and answer. However, obtaining suitable domain-specific datasets with appropriate context is often more difficult than acquiring question-answer pairs. In this paper, we investigate training QG models using synthetic contexts generated by LLMs from readily available question-answer pairs. We conduct a comprehensive study to answer critical research questions related to the performance of models trained on synthetic contexts and their potential impact on QG research and applications. Our empirical results reveal: 1) contexts are essential for QG tasks, even if they are synthetic; 2) fine-tuning smaller language models has the capability of achieving better performances as compared to prompting larger language models; and 3) synthetic context and real context could achieve comparable performances. These findings highlight the effectiveness of synthetic contexts in QG and paves the way for future advancements in the field.