Bayesian Regularization: From Tikhonov to Horseshoe

TL;DR

This paper reviews the evolution of Bayesian regularization techniques, highlighting their application in high-dimensional sparse signal recovery and covering methods from Tikhonov to horseshoe regularization.

Contribution

It provides a comprehensive review of penalty-based regularization approaches, emphasizing the progression from classical methods to modern Bayesian techniques.

Findings

Comparison of different regularization methods

Insights into the effectiveness of horseshoe regularization

Guidance on choosing regularization techniques for sparse recovery

Abstract

Bayesian regularization is a central tool in modern-day statistical and machine learning methods. Many applications involve high-dimensional sparse signal recovery problems. The goal of our paper is to provide a review of the literature on penalty-based regularization approaches, from Tikhonov (Ridge, Lasso) to horseshoe regularization.