Neural Models for Key Phrase Detection and Question Generation

TL;DR

This paper introduces a two-stage neural system for question generation from documents, combining key-phrase extraction with sequence-to-sequence question formulation, improving fluency and answerability.

Contribution

The paper presents a novel neural approach that integrates key-phrase extraction with question generation, outperforming baseline methods and enabling better dataset augmentation.

Findings

Key-phrase extraction outperforms entity-tagging and rule-based methods.

Generated questions are fluent and answerable.

System can augment reading comprehension datasets.

Abstract

We propose a two-stage neural model to tackle question generation from documents. First, our model estimates the probability that word sequences in a document are ones that a human would pick when selecting candidate answers by training a neural key-phrase extractor on the answers in a question-answering corpus. Predicted key phrases then act as target answers and condition a sequence-to-sequence question-generation model with a copy mechanism. Empirically, our key-phrase extraction model significantly outperforms an entity-tagging baseline and existing rule-based approaches. We further demonstrate that our question generation system formulates fluent, answerable questions from key phrases. This two-stage system could be used to augment or generate reading comprehension datasets, which may be leveraged to improve machine reading systems or in educational settings.