FOLLOWUPQG: Towards Information-Seeking Follow-up Question Generation

TL;DR

This paper introduces FOLLOWUPQG, a new dataset for generating follow-up questions that seek deeper understanding, highlighting the complexity and diversity of human curiosity-driven inquiries in real-world settings.

Contribution

The paper presents FOLLOWUPQG, a novel dataset of over 3,000 real-world follow-up questions from Reddit, and evaluates question generation models on this challenging benchmark.

Findings

Models produce questions that are adequate but less informative and complex than human questions.

FOLLOWUPQG dataset exhibits diverse pragmatic strategies and higher-order cognitive skills.

Current models struggle to match human-level informativeness in follow-up question generation.

Abstract

Humans ask follow-up questions driven by curiosity, which reflects a creative human cognitive process. We introduce the task of real-world information-seeking follow-up question generation (FQG), which aims to generate follow-up questions seeking a more in-depth understanding of an initial question and answer. We construct FOLLOWUPQG, a dataset of over 3K real-world (initial question, answer, follow-up question) tuples collected from a Reddit forum providing layman-friendly explanations for open-ended questions. In contrast to existing datasets, questions in FOLLOWUPQG use more diverse pragmatic strategies to seek information, and they also show higher-order cognitive skills (such as applying and relating). We evaluate current question generation models on their efficacy for generating follow-up questions, exploring how to generate specific types of follow-up questions based on step-by-step demonstrations. Our results validate FOLLOWUPQG as a challenging benchmark, as model-generated questions are adequate but far from human-raised questions in terms of informativeness and complexity.