PEOAT: Personalization-Guided Evolutionary Question Assembly for One-Shot Adaptive Testing

TL;DR

PEOAT introduces a novel personalization-guided evolutionary framework for one-shot adaptive testing, enabling efficient selection of test items tailored to individual examinees in a single step, addressing practical limitations of traditional CAT methods.

Contribution

The paper proposes PEOAT, a new evolutionary question assembly framework that incorporates personalization, cognitive insights, and diversity mechanisms for one-shot adaptive testing.

Findings

PEOAT outperforms baseline methods in test item selection accuracy.

The framework effectively balances exploration and exploitation during optimization.

Case studies reveal valuable insights into personalized test assembly.

Abstract

With the rapid advancement of intelligent education, Computerized Adaptive Testing (CAT) has attracted increasing attention by integrating educational psychology with deep learning technologies. Unlike traditional paper-and-pencil testing, CAT aims to efficiently and accurately assess examinee abilities by adaptively selecting the most suitable items during the assessment process. However, its real-time and sequential nature presents limitations in practical scenarios, particularly in large-scale assessments where interaction costs are high, or in sensitive domains such as psychological evaluations where minimizing noise and interference is essential. These challenges constrain the applicability of conventional CAT methods in time-sensitive or resourceconstrained environments. To this end, we first introduce a novel task called one-shot adaptive testing (OAT), which aims to select a fixed set of optimal items for each test-taker in a one-time selection. Meanwhile, we propose PEOAT, a Personalization-guided Evolutionary question assembly framework for One-shot Adaptive Testing from the perspective of combinatorial optimization. Specifically, we began by designing a personalization-aware initialization strategy that integrates differences between examinee ability and exercise difficulty, using multi-strategy sampling to construct a diverse and informative initial population. Building on this, we proposed a cognitive-enhanced evolutionary framework incorporating schema-preserving crossover and cognitively guided mutation to enable efficient exploration through informative signals. To maintain diversity without compromising fitness, we further introduced a diversity-aware environmental selection mechanism. The effectiveness of PEOAT is validated through extensive experiments on two datasets, complemented by case studies that uncovered valuable insights.