QASnowball: An Iterative Bootstrapping Framework for High-Quality Question-Answering Data Generation

TL;DR

QASnowball is an iterative bootstrapping framework that automatically generates high-quality question-answering data from limited seed examples, improving QA model training especially in resource-scarce scenarios.

Contribution

It introduces a novel iterative framework with modules for answer extraction, question generation, and data filtering, enhancing QA data quality and quantity through self-reseeding.

Findings

Generated data achieves comparable performance to supervised data in QA tasks.

Pre-training on generated data improves downstream QA model performance.

Framework is effective in both high-resource and medium-resource language settings.

Abstract

Recent years have witnessed the success of question answering (QA), especially its potential to be a foundation paradigm for tackling diverse NLP tasks. However, obtaining sufficient data to build an effective and stable QA system still remains an open problem. For this problem, we introduce an iterative bootstrapping framework for QA data augmentation (named QASnowball), which can iteratively generate large-scale high-quality QA data based on a seed set of supervised examples. Specifically, QASnowball consists of three modules, an answer extractor to extract core phrases in unlabeled documents as candidate answers, a question generator to generate questions based on documents and candidate answers, and a QA data filter to filter out high-quality QA data. Moreover, QASnowball can be self-enhanced by reseeding the seed set to fine-tune itself in different iterations, leading to continual improvements in the generation quality. We conduct experiments in the high-resource English scenario and the medium-resource Chinese scenario, and the experimental results show that the data generated by QASnowball can facilitate QA models: (1) training models on the generated data achieves comparable results to using supervised data, and (2) pre-training on the generated data and fine-tuning on supervised data can achieve better performance. Our code and generated data will be released to advance further work.