Fast methods for posterior inference of two-group normal-normal models

TL;DR

This paper introduces efficient algorithms for computing posterior moments in Bayesian linear regression models with normal likelihoods and priors, significantly reducing computation time compared to traditional MCMC methods.

Contribution

The paper presents a novel class of algorithms that analytically marginalize regression coefficients and use numerical integration, improving efficiency for hierarchical and random effects models.

Findings

Algorithms outperform MCMC in speed

Effective for hierarchical and mixed effects models

Demonstrated on opinion polls and COVID-19 data

Abstract

We describe a class of algorithms for evaluating posterior moments of certain Bayesian linear regression models with a normal likelihood and a normal prior on the regression coefficients. The proposed methods can be used for hierarchical mixed effects models with partial pooling over one group of predictors, as well as random effects models with partial pooling over two groups of predictors. We demonstrate the performance of the methods on two applications, one involving U.S. opinion polls and one involving the modeling of COVID-19 outbreaks in Israel using survey data. The algorithms involve analytical marginalization of regression coefficients followed by numerical integration of the remaining low-dimensional density. The dominant cost of the algorithms is an eigendecomposition computed once for each value of the outside parameter of integration. Our approach drastically reduces run times compared to state-of-the-art Markov chain Monte Carlo (MCMC) algorithms. The latter, in addition to being computationally expensive, can also be difficult to tune when applied to hierarchical models.