Sparse Horseshoe Estimation via Expectation-Maximisation

TL;DR

This paper introduces a new EM-based method for sparse Bayesian estimation using the horseshoe prior, overcoming its density function limitations and enabling efficient MAP estimation for sparse models.

Contribution

The paper develops a novel EM algorithm for horseshoe prior-based MAP estimation, extending it to generalized linear models and improving computational efficiency.

Findings

Performs comparably or better than existing methods in simulations and real data.

Offers a computationally efficient approach for sparse Bayesian estimation.

Extends to generalized linear models with simple modifications.

Abstract

The horseshoe prior is known to possess many desirable properties for Bayesian estimation of sparse parameter vectors, yet its density function lacks an analytic form. As such, it is challenging to find a closed-form solution for the posterior mode. Conventional horseshoe estimators use the posterior mean to estimate the parameters, but these estimates are not sparse. We propose a novel expectation-maximisation (EM) procedure for computing the MAP estimates of the parameters in the case of the standard linear model. A particular strength of our approach is that the M-step depends only on the form of the prior and it is independent of the form of the likelihood. We introduce several simple modifications of this EM procedure that allow for straightforward extension to generalised linear models. In experiments performed on simulated and real data, our approach performs comparable, or superior to, state-of-the-art sparse estimation methods in terms of statistical performance and computational cost.