New approach to Bayesian high-dimensional linear regression

TL;DR

This paper introduces Q-MAP, a new Bayesian linear regression method that is computationally feasible, scales well to high dimensions, and achieves near-optimal performance in noiseless settings with i.i.d. data.

Contribution

The paper proposes Q-MAP, a novel Bayesian regression scheme that combines MAP-like properties with scalable optimization and robustness to noise, addressing computational limitations of prior methods.

Findings

Achieves asymptotically optimal performance in noiseless i.i.d. settings.

Scales favorably with high-dimensional data.

Provides a robust iterative algorithm for practical implementation.

Abstract

Consider the problem of estimating parameters $X^n \in \mathbb{R}^n $, generated by a stationary process, from $m$ response variables $Y^m = AX^n+Z^m$, under the assumption that the distribution of $X^n$ is known. This is the most general version of the Bayesian linear regression problem. The lack of computationally feasible algorithms that can employ generic prior distributions and provide a good estimate of $X^n$ has limited the set of distributions researchers use to model the data. In this paper, a new scheme called Q-MAP is proposed. The new method has the following properties: (i) It has similarities to the popular MAP estimation under the noiseless setting. (ii) In the noiseless setting, it achieves the "asymptotically optimal performance" when $X^n$ has independent and identically distributed components. (iii) It scales favorably with the dimensions of the problem and therefore is applicable to high-dimensional setups. (iv) The solution of the Q-MAP optimization can be found via a proposed iterative algorithm which is provably robust to the error (noise) in the response variables.