Two Results on Low-Rank Heavy-Tailed Multiresponse Regressions

TL;DR

This paper presents new theoretical results for estimating low-rank matrices in multivariate linear models with heavy-tailed data, demonstrating near-optimal convergence rates and robustness in various quantization and response scenarios.

Contribution

It introduces robust estimators for low-rank multi-response regressions under heavy-tailed distributions, achieving near-optimal convergence rates and handling quantization.

Findings

Estimators achieve near-optimal convergence rates under heavy-tailed data.

Robust methods perform well even with only $(2+ ext{epsilon})$-order moments.

Simulations confirm theoretical results and estimator superiority.

Abstract

This paper gives two theoretical results on estimating low-rank parameter matrices for linear models with multivariate responses. We first focus on robust parameter estimation of low-rank multi-task learning with heavy-tailed data and quantization scenarios. It comprises two cases: quantization under heavy-tailed responses and quantization with both heavy-tailed covariate and response variables. For each case, our theory shows that the proposed estimator has a minmax near-optimal convergence rate. We then further investigate low-rank linear models with heavy-tailed matrix-type responses. The theory shows that when the random noise has only $(2+\epsilon)$-order moment, our robust estimator still has almost the same statistical convergence rate as that of sub-Gaussian data. Moreover, our simulation experiments confirm the correctness of theories and show the superiority of our estimators.