PS$^2$: Parameterized Control for Fine-Grained Student Proficiency Simulation

TL;DR

This paper introduces PS$^2$, a novel parameterized framework that simulates student proficiency levels more precisely by interpolating between strong and error-informed models, improving educational data analysis.

Contribution

The paper presents an unsupervised, parameterized model that calibrates student proficiency simulation by interpolating between two LLMs, enhancing control and alignment with academic performance.

Findings

Achieves finer-grained proficiency simulation compared to baselines.

Improves student behavior similarity measurement.

Enhances item difficulty prediction accuracy.

Abstract

Understanding how students with different proficiency levels respond to educational materials is a critical issue within the field of AI for Education. However, acquiring sufficient real student response data for a robust evaluation is often hindered by cost, ethics, and security constraints. Consequently, LLM-based student proficiency simulation, especially prompt-based methods, has emerged as a practical alternative under data-scarce conditions. Despite their promise, current methods still exhibit limited controllability with coarse-grained proficiency representations, high sensitivity to prompt design, and the lack of calibration with academic performance. Therefore, we propose Parameterized Student Proficiency Simulation (PS$^2$), an unsupervised and parameterized model-level framework that simulates students with different proficiencies by interpolating between a strong upper-bound LLM and a weaker, cognitive error-informed lower-bound student LLM via a hybrid ratio. Specifically, the lower-bound model is constructed by fine-tuning the weaker LM to exhibit cognitive errors when responding to educational materials. To ensure alignment with target proficiency levels, PS$^2$ further calibrates the interpolation ratio with academic performance. Experiments on two public datasets demonstrate that PS$^2$ achieves finer-grained and consistent proficiency simulation compared to existing baselines, leading to superior performance in student behavior similarity and item difficulty prediction.