An MCMC Algorithm for Estimating the Reduced RUM

TL;DR

This paper develops an MCMC algorithm for Bayesian estimation of the Reduced RUM in language assessment, incorporating correlated attributes and using a Metropolis within Gibbs sampler, validated through simulations and empirical data.

Contribution

It introduces a novel MCMC algorithm for Bayesian RRUM that handles correlated attributes and uses a saturated model with Dirichlet priors, advancing current estimation methods.

Findings

Algorithm performs well in simulations with different Q-matrix structures

Parameter estimates are comparable to those from existing R packages

The method is implemented in R for practical use

Abstract

The RRUM is a model that is frequently seen in language assessment studies. The objective of this research is to advance an MCMC algorithm for the Bayesian RRUM. The algorithm starts with estimating correlated attributes. Using a saturated model and a binary decimal conversion, the algorithm transforms possible attribute patterns to a Multinomial distribution. Along with the likelihood of an attribute pattern, a Dirichlet distribution is used as the prior to sample from the posterior. The Dirichlet distribution is constructed using Gamma distributions. Correlated attributes of examinees are generated using the inverse transform sampling. Model parameters are estimated using the Metropolis within Gibbs sampler sequentially. Two simulation studies are conducted to evaluate the performance of the algorithm. The first simulation uses a complete and balanced Q-matrix that measures 5 attributes. Comprised of 28 items and 9 attributes, the Q-matrix for the second simulation is incomplete and imbalanced. The empirical study uses the ECPE data obtained from the CDM R package. Parameter estimates from the MCMC algorithm and from the CDM R package are presented and compared. The algorithm developed in this research is implemented in R.