Multiplicative Perturbation Bounds for Multivariate Multiple Linear Regression in Schatten $p$-Norms

TL;DR

This paper extends multiplicative perturbation bounds to multivariate multiple linear regression under Schatten p-norms, providing geometric insights and analyzing the impact of sketching matrices on solution accuracy.

Contribution

It generalizes recent MLR analyses to sketched MMLR in Schatten p-norms, offering new geometric interpretations and perturbation bounds.

Findings

Derived exact and perturbed solutions using projectors.

Provided geometric interpretation of sketching matrix action.

Linked solution accuracy to principal angles between subspaces.

Abstract

Multivariate multiple linear regression (MMLR), which occurs in a number of practical applications, generalizes traditional least squares (multivariate linear regression) to multiple right-hand sides. We extend recent MLR analyses to sketched MMLR in general Schatten $p$-norms by interpreting the sketched problem as a multiplicative perturbation. Our work represents an extension of Maher's results on Schatten $p$-norms. We derive expressions for the exact and perturbed solutions in terms of projectors for easy geometric interpretation. We also present a geometric interpretation of the action of the sketching matrix in terms of relevant subspaces. We show that a key term in assessing the accuracy of the sketched MMLR solution can be viewed as a tangent of a largest principal angle between subspaces under some assumptions. Our results enable additional interpretation of the difference between an orthogonal and oblique projector with the same range.