Robust Sparse Reduced Rank Regression in High Dimensions

TL;DR

This paper introduces a robust sparse reduced rank regression method designed for high-dimensional data with heavy-tailed noise, providing theoretical error bounds and demonstrating superior performance through simulations and real data analysis.

Contribution

It develops a convex relaxation approach for robust sparse reduced rank regression, with non-asymptotic error bounds that account for heavy-tailed noise, advancing beyond existing methods focused on rank and prediction.

Findings

The method achieves error bounds under Frobenius and nuclear norms.

Convergence rates depend on the noise's tail behavior, slower for heavier tails.

Numerical studies and data analysis validate the method's effectiveness.

Abstract

We propose robust sparse reduced rank regression for analyzing large and complex high-dimensional data with heavy-tailed random noise. The proposed method is based on a convex relaxation of a rank- and sparsity-constrained non-convex optimization problem, which is then solved using the alternating direction method of multipliers algorithm. We establish non-asymptotic estimation error bounds under both Frobenius and nuclear norms in the high-dimensional setting. This is a major contribution over existing results in reduced rank regression, which mainly focus on rank selection and prediction consistency. Our theoretical results quantify the tradeoff between heavy-tailedness of the random noise and statistical bias. For random noise with bounded $(1+\delta)$th moment with $\delta \in (0,1)$, the rate of convergence is a function of $\delta$, and is slower than the sub-Gaussian-type deviation bounds; for random noise with bounded second moment, we obtain a rate of convergence as if sub-Gaussian noise were assumed. Furthermore, the transition between the two regimes is smooth. We illustrate the performance of the proposed method via extensive numerical studies and a data application.