Multidimensional Item Response Theory in the Style of Collaborative Filtering

TL;DR

This paper introduces a machine learning approach to multidimensional item response theory inspired by collaborative filtering, enabling efficient modeling and evaluation of student performance data, including large-scale sparse datasets like MOOCs.

Contribution

It proposes a general class of MIRT models based on collaborative filtering, with optimized estimation and validation techniques for large, sparse educational datasets.

Findings

Effective modeling of large-scale MOOC data.

Improved model validation using auxiliary item popularity information.

Demonstrated efficiency with simulated and real datasets.

Abstract

This paper presents a machine learning approach to multidimensional item response theory (MIRT), a class of latent factor models that can be used to model and predict student performance from observed assessment data. Inspired by collaborative filtering, we define a general class of models that includes many MIRT models. We discuss the use of penalized joint maximum likelihood (JML) to estimate individual models and cross-validation to select the best performing model. This model evaluation process can be optimized using batching techniques, such that even sparse large-scale data can be analyzed efficiently. We illustrate our approach with simulated and real data, including an example from a massive open online course (MOOC). The high-dimensional model fit to this large and sparse dataset does not lend itself well to traditional methods of factor interpretation. By analogy to recommender-system applications, we propose an alternative "validation" of the factor model, using auxiliary information about the popularity of items consulted during an open-book exam in the course.