Bayes Nets in Educational Assessment: Where Do the Numbers Come From?

TL;DR

This paper discusses the application of Bayesian inference networks in educational assessment, detailing how they integrate evidence and student models, and how MCMC techniques estimate probabilities from data, enhancing assessment accuracy.

Contribution

It introduces a framework for using Bayesian inference networks in assessment design, including methods for estimating probabilities with MCMC and special cases like IRT and latent class models.

Findings

Bayesian inference networks effectively integrate evidence and student models.

MCMC techniques can estimate probabilities from empirical data.

Numerical example demonstrates application of latent class modeling.

Abstract

As observations and student models become complex, educational assessments that exploit advances in technology and cognitive psychology can outstrip familiar testing models and analytic methods. Within the Portal conceptual framework for assessment design, Bayesian inference networks (BINs) record beliefs about students' knowledge and skills, in light of what they say and do. Joining evidence model BIN fragments- which contain observable variables and pointers to student model variables - to the student model allows one to update belief about knowledge and skills as observations arrive. Markov Chain Monte Carlo (MCMC) techniques can estimate the required conditional probabilities from empirical data, supplemented by expert judgment or substantive theory. Details for the special cases of item response theory (IRT) and multivariate latent class modeling are given, with a numerical example of the latter.