Fitting Bayesian item response models in Stata and Stan

TL;DR

This paper compares Stata's native Bayesian inference with Stan (via StataStan) for education models, showing Stan's superior flexibility, scalability, and speed, and recommends Stan as the preferred tool for Bayesian analysis in Stata.

Contribution

It demonstrates that Stan, integrated with Stata, outperforms Stata's native Bayesian methods in model flexibility, scalability, and computational efficiency for education research models.

Findings

Stan fits a broader range of models than bayesmh.

Stan is 2 to 10 times faster than bayesmh in effective sample size per second.

Jags performs better than bayesmh but not as well as Stan.

Abstract

Stata users have access to two easy-to-use implementations of Bayesian inference: Stata's native {\tt bayesmh} function and StataStan, which calls the general Bayesian engine Stan. We compare these on two models that are important for education research: the Rasch model and the hierarchical Rasch model. Stan (as called from Stata) fits a more general range of models than can be fit by {\tt bayesmh} and is also more scalable, in that it could easily fit models with at least ten times more parameters than could be fit using Stata's native Bayesian implementation. In addition, Stan runs between two and ten times faster than {\tt bayesmh} as measured in effective sample size per second: that is, compared to Stan, it takes Stata's built-in Bayesian engine twice to ten times as long to get inferences with equivalent precision. We attribute Stan's advantage in flexibility to its general modeling language, and its advantages in scalability and speed to an efficient sampling algorithm: Hamiltonian Monte Carlo using the no-U-turn sampler. In order to further investigate scalability, we also compared to the package Jags, which performed better than Stata's native Bayesian engine but not as well as StataStan. Given its advantages in speed, generality, and scalability, and that Stan is open-source and can be run directly from Stata using StataStan, we recommend that Stata users adopt Stan as their Bayesian inference engine of choice.