Scale-Invariant Robust Estimation of High-Dimensional Kronecker-Structured Matrices

TL;DR

This paper introduces a novel robust estimation method for high-dimensional Kronecker-structured matrices that effectively handles scaling ambiguities and heavy-tailed noise, improving stability and interpretability.

Contribution

The authors develop Scaled Robust Gradient Descent (SRGD) and Scaled Hard Thresholding (SHT), providing a new two-step estimation framework with proven convergence rates under challenging noise conditions.

Findings

Proposed method outperforms existing techniques in robustness and efficiency.

Convergence rates are established for various noise distributions.

Experiments demonstrate superior performance on real-world data.

Abstract

High-dimensional Kronecker-structured estimation faces a conflict between non-convex scaling ambiguities and statistical robustness. The arbitrary factor scaling distorts gradient magnitudes, rendering standard fixed-threshold robust methods ineffective. We resolve this via Scaled Robust Gradient Descent (SRGD), which stabilizes optimization by de-scaling gradients before truncation. To further enforce interpretability, we introduce Scaled Hard Thresholding (SHT) for invariant variable selection. A two-step estimation procedure, built upon robust initialization and SRGD--SHT iterative updates, is proposed for canonical matrix problems, such as trace regression, matrix GLMs, and bilinear models. The convergence rates are established for heavy-tailed predictors and noise, identifying a phase transition where optimal convergence rates recover under finite noise variance and degrade optimally for heavier tails. Experiments on simulated data and two real-world applications confirm superior robustness and efficiency of the proposed procedure.