Stronger Transformers for Neural Multi-Hop Question Generation

TL;DR

This paper introduces advanced transformer models for multi-hop question generation, outperforming previous methods by leveraging standard transformers and graph-based augmentations, with key factors like contrastive objectives influencing performance.

Contribution

The work presents a series of strong transformer models, including a graph-augmented version, that significantly improve multi-hop question generation over prior state-of-the-art methods.

Findings

Standard transformers outperform graph-based models alone.

Graph augmentations provide additional performance gains.

Auxiliary contrastive objectives and data filtering greatly impact results.

Abstract

Prior work on automated question generation has almost exclusively focused on generating simple questions whose answers can be extracted from a single document. However, there is an increasing interest in developing systems that are capable of more complex multi-hop question generation, where answering the questions requires reasoning over multiple documents. In this work, we introduce a series of strong transformer models for multi-hop question generation, including a graph-augmented transformer that leverages relations between entities in the text. While prior work has emphasized the importance of graph-based models, we show that we can substantially outperform the state-of-the-art by 5 BLEU points using a standard transformer architecture. We further demonstrate that graph-based augmentations can provide complimentary improvements on top of this foundation. Interestingly, we find that several important factors--such as the inclusion of an auxiliary contrastive objective and data filtering could have larger impacts on performance. We hope that our stronger baselines and analysis provide a constructive foundation for future work in this area.