Proof of The TAP Free Energy for High-Dimensional Linear Regression with Spherical Priors at All Temperatures

TL;DR

This paper proves that the TAP free energy accurately describes Bayesian linear regression with spherical priors across all noise levels, confirming its superiority over mean-field approximations in high-dimensional inference.

Contribution

It provides the first rigorous proof that the TAP free energy is valid for Bayesian linear regression with spherical priors at any noise level, extending previous high-noise results.

Findings

TAP free energy holds for all noise levels in Bayesian linear regression.

The global optimizer of the TAP functional is rigorously characterized.

TAP provides a more accurate variational approximation than mean-field in this setting.

Abstract

Approximate inference is central to Bayesian learning, with variational inference (VI) providing a scalable framework for posterior approximation. While mean-field VI often fails in high dimensions, the more refined Bethe approximation, equivalent to the Thouless-Anderson-Palmer (TAP) free energy in statistical physics, has long been conjectured to capture Bayes-optimal behavior. We prove that the TAP formula holds for Bayesian linear regression with a uniform spherical prior at all noise levels ($\Delta>0$), extending the result of Qiu and Sen (2023) in the high-noise regime. Our argument constructs a ridge regression functional that dominates the TAP free energy, yielding the first rigorous analysis of the global optimizer of the non-concave TAP functional for a planted inference model at an arbitrary noise level. This verifies that TAP, rather than mean-field, is the correct variational description in this setting.