Machine Learning-Assisted High-Dimensional Matrix Estimation

TL;DR

This paper introduces a machine learning-assisted optimization approach for high-dimensional matrix estimation, improving accuracy and convergence speed over classical methods.

Contribution

It develops a neural network-enhanced LADMM algorithm with proven convergence properties, applicable to covariance and precision matrix estimation.

Findings

Reparameterized LADMM converges faster than traditional LADMM.

Neural network modeling of proximal operators enhances estimation accuracy.

Method outperforms classical algorithms across various matrix structures.

Abstract

Efficient estimation of high-dimensional matrices-including covariance and precision matrices-is a cornerstone of modern multivariate statistics. Most existing studies have focused primarily on the theoretical properties of the estimators (e.g., consistency and sparsity), while largely overlooking the computational challenges inherent in high-dimensional settings. Motivated by recent advances in learning-based optimization method-which integrate data-driven structures with classical optimization algorithms-we explore high-dimensional matrix estimation assisted by machine learning. Specifically, for the optimization problem of high-dimensional matrix estimation, we first present a solution procedure based on the Linearized Alternating Direction Method of Multipliers (LADMM). We then introduce learnable parameters and model the proximal operators in the iterative scheme with neural networks, thereby improving estimation accuracy and accelerating convergence. Theoretically, we first prove the convergence of LADMM, and then establish the convergence, convergence rate, and monotonicity of its reparameterized counterpart; importantly, we show that the reparameterized LADMM enjoys a faster convergence rate. Notably, the proposed reparameterization theory and methodology are applicable to the estimation of both high-dimensional covariance and precision matrices. We validate the effectiveness of our method by comparing it with several classical optimization algorithms across different structures and dimensions of high-dimensional matrices.