A Bayesian Lasso based Sparse Learning Model

TL;DR

This paper introduces the Bayesian Lasso Sparse (BLS) model, a novel hierarchical Bayesian approach that provides sparse estimates for linear and nonlinear regression, outperforming existing methods on noisy data.

Contribution

The paper develops a new BLS model using a type-II maximum likelihood estimation, offering improved sparsity and accuracy over traditional Bayesian Lasso and related methods.

Findings

BLS produces sparse and accurate regression estimates.

BLS outperforms Relevance Vector Machine and other methods on noisy datasets.

BLS is effective for both linear and nonlinear problems.

Abstract

The Bayesian Lasso is constructed in the linear regression framework and applies the Gibbs sampling to estimate the regression parameters. This paper develops a new sparse learning model, named the Bayesian Lasso Sparse (BLS) model, that takes the hierarchical model formulation of the Bayesian Lasso. The main difference from the original Bayesian Lasso lies in the estimation procedure; the BLS method uses a learning algorithm based on the type-II maximum likelihood procedure. Opposed to the Bayesian Lasso, the BLS provides sparse estimates of the regression parameters. The BLS method is also derived for nonlinear supervised learning problems by introducing kernel functions. We compare the BLS model to the well known Relevance Vector Machine, the Fast Laplace method, the Byesian Lasso, and the Lasso, on both simulated and real data. The numerical results show that the BLS is sparse and precise, especially when dealing with noisy and irregular dataset.