Nonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests

TL;DR

This paper investigates nonlinear sequential design strategies for logistic item response theory models in computerized adaptive testing, proposing modifications to ensure consistent ability estimation across different models.

Contribution

It introduces modified adaptive design methods for logistic IRT models that guarantee consistency and asymptotic normality of ability estimators.

Findings

Maximizing item information yields consistent estimators in Rasch models.

Modified designs extend these properties to two- and three-parameter models.

Without modifications, maximum likelihood estimators may be inconsistent.

Abstract

Computerized adaptive testing is becoming increasingly popular due to advancement of modern computer technology. It differs from the conventional standardized testing in that the selection of test items is tailored to individual examinee's ability level. Arising from this selection strategy is a nonlinear sequential design problem. We study, in this paper, the sequential design problem in the context of the logistic item response theory models. We show that the adaptive design obtained by maximizing the item information leads to a consistent and asymptotically normal ability estimator in the case of the Rasch model. Modifications to the maximum information approach are proposed for the two- and three-parameter logistic models. Similar asymptotic properties are established for the modified designs and the resulting estimator. Examples are also given in the case of the two-parameter logistic model to show that without such modifications, the maximum likelihood estimator of the ability parameter may not be consistent.