A Practical Toolkit for Multilingual Question and Answer Generation

TL;DR

This paper introduces AutoQG, a multilingual question and answer generation toolkit with models in eight languages, providing both online and local options for practitioners to generate structured QA pairs easily.

Contribution

It presents AutoQG and lmqg, a comprehensive toolkit and models for multilingual question-answer generation, facilitating easy access and customization for users.

Findings

Released QA models in eight languages

Developed an online service and Python package for QAG

Enabled easy fine-tuning and evaluation of models

Abstract

Generating questions along with associated answers from a text has applications in several domains, such as creating reading comprehension tests for students, or improving document search by providing auxiliary questions and answers based on the query. Training models for question and answer generation (QAG) is not straightforward due to the expected structured output (i.e. a list of question and answer pairs), as it requires more than generating a single sentence. This results in a small number of publicly accessible QAG models. In this paper, we introduce AutoQG, an online service for multilingual QAG, along with lmqg, an all-in-one Python package for model fine-tuning, generation, and evaluation. We also release QAG models in eight languages fine-tuned on a few variants of pre-trained encoder-decoder language models, which can be used online via AutoQG or locally via lmqg. With these resources, practitioners of any level can benefit from a toolkit that includes a web interface for end users, and easy-to-use code for developers who require custom models or fine-grained controls for generation.