Learnable Graph-regularization for Matrix Decomposition

TL;DR

This paper introduces LGMD, a novel matrix decomposition method that adaptively learns graph structures to improve low-rank approximation, robustness to noise, and handling of missing data.

Contribution

LGMD is the first method to learn two graphical structures simultaneously in real-time, bridging graph regularization with probabilistic matrix decomposition.

Findings

LGMD outperforms existing methods in noisy and incomplete data scenarios.

It effectively learns true underlying data structures.

Demonstrates robustness and improved accuracy across various datasets.

Abstract

Low-rank approximation models of data matrices have become important machine learning and data mining tools in many fields including computer vision, text mining, bioinformatics and many others. They allow for embedding high-dimensional data into low-dimensional spaces, which mitigates the effects of noise and uncovers latent relations. In order to make the learned representations inherit the structures in the original data, graph-regularization terms are often added to the loss function. However, the prior graph construction often fails to reflect the true network connectivity and the intrinsic relationships. In addition, many graph-regularized methods fail to take the dual spaces into account. Probabilistic models are often used to model the distribution of the representations, but most of previous methods often assume that the hidden variables are independent and identically distributed for simplicity. To this end, we propose a learnable graph-regularization model for matrix decomposition (LGMD), which builds a bridge between graph-regularized methods and probabilistic matrix decomposition models. LGMD learns two graphical structures (i.e., two precision matrices) in real-time in an iterative manner via sparse precision matrix estimation and is more robust to noise and missing entries. Extensive numerical results and comparison with competing methods demonstrate its effectiveness.