Exploring the Psychometric Validity of AI-Generated Student Responses: A Study on Virtual Personas' Learning Motivation

TL;DR

This study investigates whether GPT-4o can generate virtual student responses that accurately reflect real student motivation, supporting its use in educational assessment validation.

Contribution

It demonstrates that GPT-4o can produce valid, structured student responses that replicate real motivational profiles for educational measurement.

Findings

GPT-4o reproduces the AMS structure

Virtual personas show distinct motivational subgroups

Factor analyses confirm validity

Abstract

This study explores whether large language models (LLMs) can simulate valid student responses for educational measurement. Using GPT -4o, 2000 virtual student personas were generated. Each persona completed the Academic Motivation Scale (AMS). Factor analyses(EFA and CFA) and clustering showed GPT -4o reproduced the AMS structure and distinct motivational subgroups.