Variational Inference for Sparse Poisson Regression

TL;DR

This paper develops a variational Bayesian approach for sparse Poisson regression, approximating the likelihood to enable efficient inference with various sparsity priors, and compares its performance to MCMC and frequentist methods.

Contribution

It introduces a novel VB method with likelihood approximation for sparse Poisson regression, enabling faster computation and effective sparsity enforcement.

Findings

VB methods closely approximate posterior distributions

Significantly faster than MCMC benchmarks

Competitive prediction performance on real data

Abstract

We have utilized the non-conjugate Variational Bayesian (VB) method for the problem of the sparse Poisson regression model. To provide approximate conjugacy in the model, the likelihood is approximated by a quadratic function, yielding conjugacy between the approximation component and the Gaussian prior on the regression coefficient. Three sparsity-enforcing priors (Laplace, Continuous Spike and Slab, and Bernoulli) are used for this problem. The proposed models are compared with each other, the associated MCMC models, and two frequentist sparse Poisson methods (LASSO and SCAD) to evaluate their estimation, prediction, and sparsity performance. In a simulation study, the proposed VB methods closely approximate the posterior parameter distribution while achieving significantly faster computation than benchmark MCMC methods. Using several benchmark count response data sets, the prediction performance of the proposed methods is evaluated in real-world applications.