Local shrinkage rules, Levy processes, and regularized regression

TL;DR

This paper introduces a Levy process-based framework for constructing prior distributions in high-dimensional regularized regression, revealing new connections and enabling efficient computation of posterior estimates.

Contribution

It generalizes local-global shrinkage rules using Levy processes, extends to large-scale p>n regression, and compares with existing methods.

Findings

New Levy process-based priors for high-dimensional regression

Framework for computing posterior means and modes under these priors

Comparative analysis with existing regularization techniques

Abstract

We use Levy processes to generate joint prior distributions, and therefore penalty functions, for a location parameter as p grows large. This generalizes the class of local-global shrinkage rules based on scale mixtures of normals, illuminates new connections among disparate methods, and leads to new results for computing posterior means and modes under a wide class of priors. We extend this framework to large-scale regularized regression problems where p>n, and provide comparisons with other methodologies.