AI-Assisted Model for Generating Multiple-Choice Questions

TL;DR

This paper presents a goal-oriented AI-human collaborative model for generating high-quality multiple-choice questions efficiently, addressing content coverage and exam security challenges in educational testing.

Contribution

It introduces a novel three-step AI-assisted prototype creation process and a one-step transformation model for generating diverse MCQ series, enhancing quality and efficiency.

Findings

Approximately 50% of generated MCQs were acceptable without editing.

Minor corrections increased acceptance rates and improved prototype quality.

The model effectively balances automation and human oversight for high-quality MCQ generation.

Abstract

Multiple-choice questions (MCQs) are widely used across diverse educational fields and levels. Well-designed MCQs should evaluate knowledge application in real-world situations. However, writing such test items in sufficient numbers is challenging and time-consuming, especially in natural science education. The problem of a sufficient number of MCQs has two aspects: content coverage and exam security. Therefore, generating test items involves two tasks: creating MCQ prototypes and transforming these prototypes into item series. In automated item generation, prototype creation aligns with template-based methods like cognitive modelling, while item expansion corresponds to example-based techniques. The aim of this research was designing the goal-oriented conceptual model of human - AI co-creation of MCQs that should meet strictly formulated quality criteria. The resulting three-step model for creating MCQ prototypes distributed prompts between several AIs, with human revision of responses for each prompt before setting the next one. To transform the MCQ prototype into an MCQ series, a one-step model was developed in which multiple new items are generated simultaneously. These items assess the same learning outcome but are not simple rephrasings of the prototype or of one another. Based on human and automated evaluation, approximately half of the output MCQs were acceptable without editing. Minor corrections of initially rejected test items allowed for a moderate increase in acceptance of MCQs in series and a significant improvement of MCQ-prototypes.