Bayesian Inference for Logistic Regression Models Using Sequential Posterior Simulation

TL;DR

This paper introduces a simple, efficient sequential posterior simulation method for Bayesian logistic regression that outperforms traditional MCMC approaches, especially when implemented on GPUs, and provides reliable standard errors and marginal likelihood estimates.

Contribution

It presents a novel SPS method for Bayesian logistic regression that is easier to implement, highly parallelizable, and more efficient than existing techniques.

Findings

SPS method is simpler and requires only likelihood evaluation.

GPU implementation significantly speeds up computation.

Provides accurate standard errors and marginal likelihoods.

Abstract

The logistic specification has been used extensively in non-Bayesian statistics to model the dependence of discrete outcomes on the values of specified covariates. Because the likelihood function is globally weakly concave estimation by maximum likelihood is generally straightforward even in commonly arising applications with scores or hundreds of parameters. In contrast Bayesian inference has proven awkward, requiring normal approximations to the likelihood or specialized adaptations of existing Markov chain Monte Carlo and data augmentation methods. This paper approaches Bayesian inference in logistic models using recently developed generic sequential posterior simulaton (SPS) methods that require little more than the ability to evaluate the likelihood function. Compared with existing alternatives SPS is much simpler, and provides numerical standard errors and accurate approximations of marginal likelihoods as by-products. The SPS algorithm for Bayesian inference is amenable to massively parallel implementation, and when implemented using graphical processing units it is more efficient than existing alternatives. The paper demonstrates these points by means of several examples.