Fast Rank Reduction for Non-negative Matrices via Mean Field Theory

TL;DR

This paper introduces a fast, convex optimization-based method for non-negative matrix rank reduction using mean-field theory, outperforming traditional NMF in speed while maintaining low approximation error.

Contribution

It presents a novel mean-field approximation approach for matrix rank reduction that allows for closed-form solutions and improved computational efficiency.

Findings

Faster than NMF and lraNMF methods.

Achieves competitive low-rank approximation accuracy.

Validated on synthetic and real-world datasets.

Abstract

We propose an efficient matrix rank reduction method for non-negative matrices, whose time complexity is quadratic in the number of rows or columns of a matrix. Our key insight is to formulate rank reduction as a mean-field approximation by modeling matrices via a log-linear model on structured sample space, which allows us to solve the rank reduction as convex optimization. The highlight of this formulation is that the optimal solution that minimizes the KL divergence from a given matrix can be analytically computed in a closed form. We empirically show that our rank reduction method is faster than NMF and its popular variant, lraNMF, while achieving competitive low rank approximation error on synthetic and real-world datasets.