"What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation

TL;DR

This paper introduces a type-controlled inquisitive question generation framework that produces diverse, high-quality questions aligned with specific types, and employs a ranking strategy to select the most inquisitive questions, rivaling human performance.

Contribution

It presents a novel type-controlled question generation method, combined with a ranking approach for selecting the most inquisitive questions, validated through extensive automatic and human evaluations.

Findings

Generated questions match specific types accurately.

The pairwise ranker effectively selects questions with high syntax, semantics, and inquisitiveness.

Human evaluation shows questions rival human-written ones in quality.

Abstract

We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and finetune models for type-controlled question generation. Empirical results demonstrate that we can generate a variety of questions that adhere to specific types while drawing from the source texts. We also investigate strategies for selecting a single question from a generated set, considering both an informative vs.~inquisitive question classifier and a pairwise ranker trained from a small set of expert annotations. Question selection using the pairwise ranker yields strong results in automatic and manual evaluation. Our human evaluation assesses multiple aspects of the generated questions, finding that the ranker chooses questions with the best syntax (4.59), semantics (4.37), and inquisitiveness (3.92) on a scale of 1-5, even rivaling the performance of human-written questions.