Robust Subspace-Constrained Quadratic Models for Low-Dimensional Structure Learning

TL;DR

This paper introduces a robust subspace-constrained quadratic model for low-dimensional structure learning that handles diverse noise types and improves robustness and accuracy in high-dimensional data analysis.

Contribution

It extends the SQMF framework to accommodate various noise distributions and develops an efficient gradient-based algorithm with theoretical analysis.

Findings

Outperforms existing methods in robustness and accuracy

Handles heavy-tailed and light-tailed noise effectively

Provides sensitivity analysis of loss functions under different noise conditions

Abstract

In this paper, we propose a robust subspace-constrained quadratic model (SCQM) for learning low-dimensional structure from high-dimensional data. Building upon the subspace-constrained quadratic matrix factorization (SQMF) framework, the proposed model accommodates a broad class of noise distributions, including generalized Gaussian and radial Laplace models. This generalization enables reliable performance under both heavy-tailed and light-tailed noise, thereby substantially enhancing robustness across diverse data regimes. To efficiently address the resulting nonconvex optimization problem, we develop a gradient-based algorithm equipped with a backtracking line-search strategy that ensures stable and efficient convergence. In addition, we present a sensitivity analysis of the $\ell_p^p$ and $\ell_2$ loss functions, elucidating their distinct behaviors under varying noise characteristics. Extensive numerical experiments corroborate the theoretical analysis and demonstrate that the proposed approach consistently outperforms existing methods in terms of robustness and reconstruction accuracy.