Computerized adaptive testing: implementation issues

TL;DR

This paper discusses the implementation challenges of Computerized Adaptive Testing systems, compares them with traditional computer-based tests, and presents strategies and a system development process to improve their effectiveness.

Contribution

It introduces an item difficulty estimation technique and compares item selection strategies to address item exposure issues in CAT systems.

Findings

Comparison of CAT and CBT advantages and disadvantages

Evaluation of item selection strategies for exposure control

Development of a CAT system with an item difficulty estimation method

Abstract

One of the fastest evolving field among teaching and learning research is students' performance evaluation. Computer based testing systems are increasingly adopted by universities. However, the implementation and maintenance of such a system and its underlying item bank is a challenge for an inexperienced tutor. Therefore, this paper discusses the advantages and disadvantages of Computer Adaptive Test (CAT) systems compared to Computer Based Test systems. Furthermore, a few item selection strategies are compared in order to overcome the item exposure drawback of such systems. The paper also presents our CAT system along its development steps. Besides, an item difficulty estimation technique is presented based on data taken from our self-assessment system.