Structured Variational $D$-Decomposition for Accurate and Stable Low-Rank Approximation

TL;DR

This paper introduces the $D$-decomposition, a variational matrix factorization method that improves low-rank approximation accuracy and stability, especially in noisy or sparse data scenarios.

Contribution

It proposes a novel non-orthogonal matrix factorization framework with theoretical guarantees and demonstrates superior performance over traditional methods in various benchmarks.

Findings

Enhanced reconstruction accuracy on benchmark datasets

Stable solutions with perturbation analysis

Effective handling of sparsity and noise

Abstract

We introduce the $D$-decomposition, a non-orthogonal matrix factorization of the form $A \approx P D Q$, where $P \in \mathbb{R}^{n \times k}$, $D \in \mathbb{R}^{k \times k}$, and $Q \in \mathbb{R}^{k \times n}$. The decomposition is defined variationally by minimizing a regularized Frobenius loss, allowing control over rank, sparsity, and conditioning. Unlike algebraic factorizations such as LU or SVD, it is computed by alternating minimization. We establish existence and perturbation stability of the solution and show that each update has complexity $\mathcal{O}(n^2k)$. Benchmarks against truncated SVD, CUR, and nonnegative matrix factorization show improved reconstruction accuracy on MovieLens, MNIST, Olivetti Faces, and gene expression matrices, particularly under sparsity and noise.