Multiple Regression for Matrix and Vector Predictors: Models, Theory, Algorithms, and Beyond

TL;DR

This paper introduces a unified regularized regression framework for predicting matrix and vector variables, establishing theoretical consistency and demonstrating superior empirical performance with an efficient algorithm.

Contribution

It develops a novel class of regularized models for matrix and vector prediction, with theoretical guarantees and a scalable optimization algorithm.

Findings

Proposed models achieve higher prediction accuracy.

The estimator is consistent under nuclear and L1 norm penalties.

The algorithm outperforms existing solvers in efficiency.

Abstract

Matrix regression plays an important role in modern data analysis due to its ability to handle complex relationships involving both matrix and vector variables. We propose a class of regularized regression models capable of predicting both matrix and vector variables, accommodating various regularization techniques tailored to the inherent structures of the data. We establish the consistency of our estimator when penalizing the nuclear norm of the matrix variable and the $\ell_1$ norm of the vector variable. To tackle the general regularized regression model, we propose a unified framework based on an efficient preconditioned proximal point algorithm. Numerical experiments demonstrate the superior estimation and prediction accuracy of our proposed estimator, as well as the efficiency of our algorithm compared to the state-of-the-art solvers.