Compound Poisson Processes, Latent Shrinkage Priors and Bayesian Nonconvex Penalization

TL;DR

This paper introduces a Bayesian nonconvex penalization framework using compound Poisson subordinators for sparse learning, with algorithms for simultaneous estimation of model parameters, demonstrating effectiveness in high-dimensional data.

Contribution

It proposes a novel hierarchical Bayesian model with compound Poisson subordinators as sparsity-inducing penalties, along with ECME algorithms for parameter estimation.

Findings

Effective in high-dimensional regression tasks

Feasible and efficient estimation algorithms

Demonstrated advantages of nonconvex penalties in sparse learning

Abstract

In this paper we discuss Bayesian nonconvex penalization for sparse learning problems. We explore a nonparametric formulation for latent shrinkage parameters using subordinators which are one-dimensional L\'{e}vy processes. We particularly study a family of continuous compound Poisson subordinators and a family of discrete compound Poisson subordinators. We exemplify four specific subordinators: Gamma, Poisson, negative binomial and squared Bessel subordinators. The Laplace exponents of the subordinators are Bernstein functions, so they can be used as sparsity-inducing nonconvex penalty functions. We exploit these subordinators in regression problems, yielding a hierarchical model with multiple regularization parameters. We devise ECME (Expectation/Conditional Maximization Either) algorithms to simultaneously estimate regression coefficients and regularization parameters. The empirical evaluation of simulated data shows that our approach is feasible and effective in high-dimensional data analysis.