Posterior Impropriety of some Sparse Bayesian Learning Models

TL;DR

This paper demonstrates that certain sparse Bayesian learning models using improper priors can result in improper posteriors, highlighting a potential issue in their theoretical foundation.

Contribution

It identifies specific models where improper priors cause posterior impropriety, providing critical insights into model validity and implementation.

Findings

Some models with improper priors lead to improper posteriors

Improper priors can compromise the validity of Bayesian inference

Highlights the need for proper priors in certain sparse Bayesian models

Abstract

Sparse Bayesian learning models are typically used for prediction in datasets with significantly greater number of covariates than observations. Such models often take a reproducing kernel Hilbert space (RKHS) approach to carry out the task of prediction and can be implemented using either proper or improper priors. In this article we show that a few sparse Bayesian learning models in the literature, when implemented using improper priors, lead to improper posteriors.