Misspecifying non-compensatory as compensatory IRT: analysis of estimated skills and variance

TL;DR

This paper investigates the effects of misspecifying non-compensatory IRT models as compensatory, revealing both underestimation and overestimation of skills and analyzing how variance estimates are affected.

Contribution

It provides a theoretical analysis of both underestimation and overestimation phenomena in misspecified IRT models, including variance implications.

Findings

Underestimation of higher skills in misspecified models

Overestimation of skills around the origin

Variance of estimates is affected by model misspecification

Abstract

Multidimensional item response theory is a statistical test theory used to estimate the latent skills of learners and the difficulty levels of problems based on test results. Both compensatory and non-compensatory models have been proposed in the literature. Previous studies have revealed the substantial underestimation of higher skills when the non-compensatory model is misspecified as the compensatory model. However, the underlying mechanism behind this phenomenon has not been fully elucidated. It remains unclear whether overestimation also occurs and whether issues arise regarding the variance of the estimated parameters. In this paper, we aim to provide a comprehensive understanding of both underestimation and overestimation through a theoretical approach. In addition to the previously identified underestimation of the skills, we newly discover that the overestimation of skills occurs around the origin. Furthermore, we investigate the extent to which the asymptotic variance of the estimated parameters differs when considering model misspecification compared to when it is not taken into account.