Item Parameter Recovery for the Two-Parameter Testlet Model with Different Estimation Methods

TL;DR

This study compares the effectiveness of MCMC, MMLE, and WLSMV estimation methods in recovering parameters of the 2PL testlet model, revealing no significant differences but highlighting convergence issues with limited-information methods under certain conditions.

Contribution

It provides a comprehensive simulation comparison of three estimation methods for the 2PL testlet model, including real data application, which was lacking in prior research.

Findings

No significant difference in parameter recovery among the three methods.

WLSMV and MCMC had convergence issues with small sample sizes and testlet effects.

MCMC remained stable regardless of sample size and testlet variance.

Abstract

The testlet model is a popular statistical approach widely used by researchers and practitioners to address local item dependence (LID), a violation of the local independence assumption in item response theory (IRT) which can cause various deleterious psychometric consequences. Same as other psychometric models, the utility of the testlet model relies heavily on accurate estimation of its model parameters. The two-parameter logistic (2PL) testlet model has only been systematically investigated in the psychometric literature regarding its model parameter recovery with one full information estimation methods, namely Markov chain Monte Carlo (MCMC) method, although there are other estimation methods available such as marginal maximum likelihood estimation (MMLE) and limited information estimation methods. In the current study, a comprehensive simulation study was conducted to investigate how MCMC, MMLE, and one limited information estimation method (WLSMV), all implemented in Mplus, recovered the item parameters and the testlet variance parameter of the 2PL testlet model. The manipulated factors were sample size and testlet effect magnitude, and parameter recovery were evaluated with bias, standard error, and root mean square error. We found that there were no statistically significant differences regarding parameter recovery between the three methods. When both sample size and magnitude of testlet variance were small, both WLSMV and MCMC had convergence issues, which did not occur to MCMC regardless of sample size and testlet variance. A real dataset from a high-stakes test was used to demonstrate the estimation of the 2PL testlet model with the three estimation methods. Keywords: IRT, testlet model, estimation, full-information, limited-information.