Generating Adequate Distractors for Multiple-Choice Questions

TL;DR

This paper introduces a comprehensive method combining linguistic and knowledge-based techniques to automatically generate multiple-choice questions with adequate distractors, validated on SAT reading test data.

Contribution

It presents a novel integrated approach for automatic distractor generation for MCQs, achieving high adequacy rates validated by human evaluation.

Findings

At least one adequate distractor per MCQ

84% of MCQs have three adequate distractors

Effective on SAT reading test dataset

Abstract

This paper presents a novel approach to automatic generation of adequate distractors for a given question-answer pair (QAP) generated from a given article to form an adequate multiple-choice question (MCQ). Our method is a combination of part-of-speech tagging, named-entity tagging, semantic-role labeling, regular expressions, domain knowledge bases, word embeddings, word edit distance, WordNet, and other algorithms. We use the US SAT (Scholastic Assessment Test) practice reading tests as a dataset to produce QAPs and generate three distractors for each QAP to form an MCQ. We show that, via experiments and evaluations by human judges, each MCQ has at least one adequate distractor and 84\% of MCQs have three adequate distractors.