Equivalence of state equations from different methods in High-dimensional Regression

TL;DR

This paper demonstrates that various state equations used in high-dimensional statistics, derived from different methods, are fundamentally equivalent and can be transformed into a unified form, revealing a deeper underlying structure.

Contribution

It provides a unified framework showing the equivalence of state equations from multiple methods in high-dimensional regression analysis.

Findings

Different state equations are essentially equivalent after parameter transformations.

The equivalence links various methods like AMP and CGMT.

This insight may lead to a deeper understanding of high-dimensional statistical models.

Abstract

State equations (SEs) were firstly introduced in the approximate message passing (AMP) to describe the mean square error (MSE) in compressed sensing. Since then a set of state equations have appeared in studies of logistic regression, robust estimator and other high-dimensional statistics problems. Recently, a convex Gaussian min-max theorem (CGMT) approach was proposed to study high-dimensional statistic problems accompanying with another set of different state equations. This paper provides a uniform viewpoint on these methods and shows the equivalence of their reduction forms, which causes that the resulting SEs are essentially equivalent and can be converted into the same expression through parameter transformations. Combining these results, we show that these different state equations are derived from several equivalent reduction forms. We believe that this equivalence will shed light on discovering a deeper structure in high-dimensional statistics.