When Assessment Theory Meets Generative AI: Reimagining SBA Design in Medical Education

TL;DR

This paper introduces a new framework for integrating AI into medical education assessments, balancing quality, efficiency, and educational value.

Contribution

The Co-Created SBA Design (CCSD) framework reimagines assessment theory by integrating GenAI within Van der Vleuten’s Utility Index.

Findings

The CCSD framework maintains balance across validity, reliability, and cost-efficiency in AI-assisted assessments.

GenAI is positioned as a collaborative partner in item-writing, enhancing educational innovation.

The triadic model supports sustainable integration of AI in medical education assessments.

Abstract

Current evaluations of generative artificial intelligence (GenAI) in item-writing within medical education often concentrate on isolated dimensions such as validity or reliability, overlooking the broader theoretical foundations that underpin a trustworthy assessment design. This narrow emphasis risks oversimplifying GenAI’s role and obscuring how its adoption reshapes the relationship between quality, efficiency, and educational value. To address this complexity, this paper presents the Co-Created SBA Design (CCSD) framework, which reconceptualises assessment theory for the GenAI era through the lens of Van der Vleuten’s Utility Index. The framework offers a coherent structure for integrating GenAI into Single Best Answer development, maintaining equilibrium across the Utility Index dimensions while redefining collaboration among educators, higher education institutions, and GenAI, a technological partner that enriches the item-writing process. Within this triadic model, each contributor plays a distinct yet complementary role in sustaining assessment quality. Collectively, their interaction ensures that validity, reliability, educational impact, acceptability, and cost-efficiency remain balanced, supporting both educational integrity and sustainable innovation in medical education.

Figures