A Feasibility Study of Answer-Agnostic Question Generation for Education

TL;DR

This study explores the use of answer-agnostic question generation models for educational texts, showing that summaries improve question relevance and acceptability, with automatic summarization serving as a practical alternative.

Contribution

It demonstrates that providing summaries enhances question relevance and acceptability in answer-agnostic question generation for textbooks.

Findings

Summaries increase question acceptability from 33% to 83%.

Automatic summarization can substitute human summaries effectively.

Errors mainly stem from irrelevant or uninterpretable questions.

Abstract

We conduct a feasibility study into the applicability of answer-agnostic question generation models to textbook passages. We show that a significant portion of errors in such systems arise from asking irrelevant or uninterpretable questions and that such errors can be ameliorated by providing summarized input. We find that giving these models human-written summaries instead of the original text results in a significant increase in acceptability of generated questions (33% $\rightarrow$ 83%) as determined by expert annotators. We also find that, in the absence of human-written summaries, automatic summarization can serve as a good middle ground.