Robust Regularized Low-Rank Matrix Models for Regression and Classification

TL;DR

This paper introduces a robust matrix variate regression framework with a rank constraint and regularization, providing convergence guarantees and minimax optimality, validated through simulations and real data.

Contribution

It proposes a novel matrix regression model with a rank constraint and regularization, along with a convergence-guaranteed algorithm, advancing high-dimensional noisy data analysis.

Findings

Algorithm converges with estimation errors of order O(1/√n)

Method attains minimax rate in estimation accuracy

Validated effectiveness on simulations and real image data

Abstract

While matrix variate regression models have been studied in many existing works, classical statistical and computational methods for the analysis of the regression coefficient estimation are highly affected by high dimensional and noisy matrix-valued predictors. To address these issues, this paper proposes a framework of matrix variate regression models based on a rank constraint, vector regularization (e.g., sparsity), and a general loss function with three special cases considered: ordinary matrix regression, robust matrix regression, and matrix logistic regression. We also propose an alternating projected gradient descent algorithm. Based on analyzing our objective functions on manifolds with bounded curvature, we show that the algorithm is guaranteed to converge, all accumulation points of the iterates have estimation errors in the order of $O(1/\sqrt{n})$ asymptotically and substantially attaining the minimax rate. Our theoretical analysis can be applied to general optimization problems on manifolds with bounded curvature and can be considered an important technical contribution to this work. We validate the proposed method through simulation studies and real image data examples.