Difficulty-Controllable Cloze Question Distractor Generation

TL;DR

This paper introduces a novel framework for generating high-quality, difficulty-controllable distractors for multiple-choice cloze questions, leveraging data augmentation and multitask learning to improve difficulty alignment.

Contribution

It presents a new dataset creation process and a multitask learning model that enables controllable difficulty in distractor generation, addressing limitations of existing methods.

Findings

The method produces distractors that better match human difficulty perception.

It outperforms GPT-4o in generating difficulty-aligned distractors.

The approach effectively generates diverse and plausible distractors across difficulty levels.

Abstract

Multiple-choice cloze questions are commonly used to assess linguistic proficiency and comprehension. However, generating high-quality distractors remains challenging, as existing methods often lack adaptability and control over difficulty levels, and the absence of difficulty-annotated datasets further hinders progress. To address these issues, we propose a novel framework for generating distractors with controllable difficulty by leveraging both data augmentation and a multitask learning strategy. First, to create a high-quality, difficulty-annotated dataset, we introduce a two-way distractor generation process to produce diverse and plausible distractors. These candidates are filtered and then categorized by difficulty using an ensemble QA system. Second, this newly created dataset is used to train a difficulty-controllable generation model via multitask learning. Experimental results demonstrate that our method generates high-quality distractors across difficulty levels and substantially outperforms GPT-4o in aligning distractor difficulty with human perception.