Multi-Task Learning with Language Modeling for Question Generation

TL;DR

This paper introduces a multi-task learning approach combining answer-aware question generation with language modeling, resulting in improved question quality and state-of-the-art performance on benchmark datasets.

Contribution

It proposes a hierarchical multi-task learning model that integrates language modeling to enhance answer-aware question generation.

Findings

Achieves state-of-the-art results on SQuAD and MARCO datasets.

Generated questions are more coherent and fluent according to human evaluation.

Multi-task learning improves encoder representations for question generation.

Abstract

This paper explores the task of answer-aware questions generation. Based on the attention-based pointer generator model, we propose to incorporate an auxiliary task of language modeling to help question generation in a hierarchical multi-task learning structure. Our joint-learning model enables the encoder to learn a better representation of the input sequence, which will guide the decoder to generate more coherent and fluent questions. On both SQuAD and MARCO datasets, our multi-task learning model boosts the performance, achieving state-of-the-art results. Moreover, human evaluation further proves the high quality of our generated questions.