Sparse regression with highly correlated predictors

TL;DR

This paper introduces the clustering removal algorithm (CRA) to improve sparse regression with highly correlated, clustered predictors, demonstrating that CRA enhances recovery by satisfying the restricted isometry property under certain conditions.

Contribution

The paper proposes CRA, a novel method to decorrelate clustered predictors in sparse regression, ensuring RIP conditions and improving recovery performance.

Findings

CRA effectively decorrelates clustered predictors

CRA satisfies RIP with high probability under certain assumptions

Empirical results show CRA outperforms existing methods in correlated settings

Abstract

We consider a linear regression $y=X\beta+u$ where $X\in\mathbb{\mathbb{{R}}}^{n\times p}$, $p\gg n,$ and $\beta$ is $s$-sparse. Motivated by examples in financial and economic data, we consider the situation where $X$ has highly correlated and clustered columns. To perform sparse recovery in this setting, we introduce the \emph{clustering removal algorithm} (CRA), that seeks to decrease the correlation in $X$ by removing the cluster structure without changing the parameter vector $\beta$. We show that as long as certain assumptions hold about $X$, the decorrelated matrix will satisfy the restricted isometry property (RIP) with high probability. We also provide examples of the empirical performance of CRA and compare it with other sparse recovery techniques.