Efficient expectation propagation for posterior approximation in high-dimensional probit models

TL;DR

This paper introduces an efficient expectation propagation method for approximating the posterior in high-dimensional Bayesian probit regression, significantly reducing computational costs and enabling practical application in large-scale problems.

Contribution

It adapts extended multivariate skew-normal distribution results to develop a scalable EP algorithm with linear per-iteration complexity for high-dimensional Bayesian probit models.

Findings

EP implementation scales linearly with covariate number

Method is computationally feasible in high-dimensional settings

Simulation studies demonstrate improved efficiency

Abstract

Bayesian binary regression is a prosperous area of research due to the computational challenges encountered by currently available methods either for high-dimensional settings or large datasets, or both. In the present work, we focus on the expectation propagation (EP) approximation of the posterior distribution in Bayesian probit regression under a multivariate Gaussian prior distribution. Adapting more general derivations in Anceschi et al. (2023), we show how to leverage results on the extended multivariate skew-normal distribution to derive an efficient implementation of the EP routine having a per-iteration cost that scales linearly in the number of covariates. This makes EP computationally feasible also in challenging high-dimensional settings, as shown in a detailed simulation study.