Precision education: A Bayesian nonparametric approach for handling item and examinee heterogeneity in assessment data

TL;DR

This paper introduces a Bayesian nonparametric model for educational assessment that captures heterogeneity among questions and examinees, providing identifiable, asymptotically consistent parameter estimates and a practical sampling algorithm.

Contribution

It presents a novel nonparametric Bayesian IRT model with clustering at question and examinee levels, including an identifiable structure and efficient inference methods.

Findings

Model is identifiable and asymptotically consistent.

Parameters can be estimated at a root n rate.

Application to real data demonstrates practical utility.

Abstract

We propose a novel nonparametric Bayesian IRT model in this paper by introducing the clustering effect at question level and further assume heterogeneity at examinee level under each question cluster, characterized by the mixture of Binomial distributions. The main contribution of this work is threefold: (1) We demonstrate that the model is identifiable. (2) The clustering effect can be captured asymptotically and the parameters of interest that measure the proficiency of examinees in solving certain questions can be estimated at a root n rate (up to a log term). (3) We present a tractable sampling algorithm to obtain valid posterior samples from our proposed model. We evaluate our model via a series of simulations as well as apply it to an English assessment data. This data analysis example nicely illustrates how our model can be used by test makers to distinguish different types of students and aid in the design of future tests.