The TAP free energy for high-dimensional linear regression

TL;DR

This paper rigorously establishes the TAP approximation for Bayesian linear regression with spherical prior in high-dimensional settings, connecting spin glass theory to statistical inference.

Contribution

It provides a rigorous derivation of the TAP free energy representation in high-dimensional Bayesian linear regression with spherical prior, confirming a prior conjecture.

Findings

TAP approximation is valid in high-dimensional Bayesian linear regression.

The work confirms a conjecture by Krzakala et al. (2014).

Provides a variational representation for the log-normalizing constant.

Abstract

We derive a variational representation for the log-normalizing constant of the posterior distribution in Bayesian linear regression with a uniform spherical prior and an i.i.d. Gaussian design. We work under the "proportional" asymptotic regime, where the number of observations and the number of features grow at a proportional rate. This rigorously establishes the Thouless-Anderson-Palmer (TAP) approximation arising from spin glass theory, and proves a conjecture of Krzakala et. al. (2014) in the special case of the spherical prior.