DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking

TL;DR

DisGeM introduces a simple, effective framework for generating distractors for multiple-choice questions using pre-trained language models, eliminating the need for dataset-specific training.

Contribution

It presents a novel two-stage distractor generation framework leveraging only pre-trained models, outperforming previous methods without additional training.

Findings

Outperforms previous distractor generation methods

Produces more effective and engaging distractors according to human evaluation

Does not require dataset-specific training or fine-tuning

Abstract

Recent advancements in Natural Language Processing (NLP) have impacted numerous sub-fields such as natural language generation, natural language inference, question answering, and more. However, in the field of question generation, the creation of distractors for multiple-choice questions (MCQ) remains a challenging task. In this work, we present a simple, generic framework for distractor generation using readily available Pre-trained Language Models (PLMs). Unlike previous methods, our framework relies solely on pre-trained language models and does not require additional training on specific datasets. Building upon previous research, we introduce a two-stage framework consisting of candidate generation and candidate selection. Our proposed distractor generation framework outperforms previous methods without the need for training or fine-tuning. Human evaluations confirm that our approach produces more effective and engaging distractors. The related codebase is publicly available at https://github.com/obss/disgem.