Matrix Decomposition Perspective for Accuracy Assessment of Item Response Theory

TL;DR

This paper evaluates the accuracy of item response theory by comparing reconstructed response matrices to observed data using matrix decomposition techniques, revealing comparable performance with SVD and insights into approximation quality.

Contribution

It introduces a matrix decomposition perspective to assess the accuracy of item response theory estimates, comparing them with low-rank matrix approximations.

Findings

SVD and matrix decomposition perform similarly on complete matrices.

Reconstructed matrices' closeness lies between low-rank approximations with k= and k=2.

Performance measured by root mean squared error and accuracy.

Abstract

The item response theory obtains the estimates and their confidence intervals for parameters of abilities of examinees and difficulties of problems by using the observed item response matrix consisting of 0/1 value elements. Many papers discuss the performance of the estimates. However, this paper does not. Using the maximum likelihood estimates, we can reconstruct the estimated item response matrix. Then we can assess the accuracy of this reconstructed matrix to the observed response matrix from the matrix decomposition perspective. That is, this paper focuses on the performance of the reconstructed response matrix. To compare the performance of the item response theory with others, we provided the two kinds of low rank response matrix by approximating the observed response matrix; one is the matrix via the singular value decomposition method when the response matrix is a complete matrix, and the other is the matrix via the matrix decomposition method when the response matrix is an incomplete matrix. We have, firstly, found that the performance of the singular value decomposition method and the matrix decomposition method is almost the same when the response matrix is a complete matrix. Here, the performance is measured by the closeness between the two matrices using the root mean squared errors and the accuracy. Secondary, we have seen that the closeness of the reconstructed matrix obtained from the item response theory to the observed matrix is located between the two approximated low rank response matrices obtained from the matrix decomposition method of k= and k=2 to the observed matrix, where k indicates the first k columns use in the decomposed matrices.