Constructing Cloze Questions Generatively

TL;DR

This paper introduces CQG, a neural network-based method that automatically generates high-quality cloze questions with multigram distractors from articles, leveraging WordNet and sense disambiguation for improved accuracy.

Contribution

The paper presents a novel generative approach, CQG, that combines sense disambiguation, lexical databases, and transformer models to produce superior cloze questions with diverse distractors.

Findings

Outperforms state-of-the-art methods in automatic metrics

Human judges rate generated distractors as high quality

Significantly improves the effectiveness of cloze question generation

Abstract

We present a generative method called CQG for constructing cloze questions from a given article using neural networks and WordNet, with an emphasis on generating multigram distractors. Built on sense disambiguation, text-to-text transformation, WordNet's synset taxonomies and lexical labels, CQG selects an answer key for a given sentence, segments it into a sequence of instances, generates instance-level distractor candidates (IDCs) using a transformer and sibling synsets.It then removes inappropriate IDCs, ranks the remaining IDCs based on contextual embedding similarities, as well as synset and lexical relatedness, forms distractor candidates by combinatorially replacing instances with the corresponding top-ranked IDCs, and checks if they are legitimate phrases. Finally, it selects top-ranked distractor candidates based on contextual semantic similarities to the answer key. Experiments show that this method significantly outperforms SOTA results. Human judges also confirm the high qualities of the generated distractors.