Ultimate P\'olya Gamma Samplers -- Efficient MCMC for possibly imbalanced binary and categorical data

TL;DR

This paper introduces efficient Pólya-Gamma based Gibbs sampling algorithms for Bayesian binary and categorical data models, improving sampling efficiency and accessibility for complex logistic regression frameworks.

Contribution

It develops novel latent variable representations and marginal data augmentation strategies for Pólya-Gamma models, enhancing sampling efficiency and extending applicability.

Findings

Improved sampling efficiency demonstrated through simulations

Applicable to complex models like state space frameworks

Enhanced accessibility for Bayesian binary and categorical modeling

Abstract

Modeling binary and categorical data is one of the most commonly encountered tasks of applied statisticians and econometricians. While Bayesian methods in this context have been available for decades now, they often require a high level of familiarity with Bayesian statistics or suffer from issues such as low sampling efficiency. To contribute to the accessibility of Bayesian models for binary and categorical data, we introduce novel latent variable representations based on P\'olya-Gamma random variables for a range of commonly encountered logistic regression models. From these latent variable representations, new Gibbs sampling algorithms for binary, binomial, and multinomial logit models are derived. All models allow for a conditionally Gaussian likelihood representation, rendering extensions to more complex modeling frameworks such as state space models straightforward. However, sampling efficiency may still be an issue in these data augmentation based estimation frameworks. To counteract this, novel marginal data augmentation strategies are developed and discussed in detail. The merits of our approach are illustrated through extensive simulations and real data applications.