Generative AI for Multiple Choice STEM Assessments

TL;DR

This paper investigates using generative AI to create plausible distractors for STEM multiple choice assessments, enhancing assessment quality while reducing creation effort.

Contribution

It introduces a novel approach to generate credible incorrect options in STEM assessments using specialized semantic handling and prompt engineering.

Findings

AI can generate high-quality distractors that maintain academic rigor

Semantic packages improve the accuracy of mathematical content in AI outputs

The approach reduces time and effort in assessment material creation

Abstract

Artificial intelligence (AI) technology enables a range of enhancements in computer-aided instruction, from accelerating the creation of teaching materials to customizing learning paths based on learner outcomes. However, ensuring the mathematical accuracy and semantic integrity of generative AI output remains a significant challenge, particularly in Science, Technology, Engineering and Mathematics (STEM) disciplines. In this study, we explore the use of generative AI in which "hallucinations", typically viewed as undesirable inaccuracies, can instead serve a pedagogical purpose. Specifically, we investigate the generation of plausible but incorrect alternatives for multiple choice assessments, where credible distractors are essential for effective assessment design. We describe the Moebius platform for online instruction, with particular focus on its architecture for handling mathematical elements through specialized semantic packages that support dynamic, parameterized STEM content. We examine methods for crafting prompts that interact effectively with these mathematical semantics to guide the AI in generating high-quality multiple choice distractors. Finally, we demonstrate how this approach reduces the time and effort associated with creating robust teaching materials while maintaining academic rigor and assessment validity.