A Column-Wise Update Algorithm for Sparse Stochastic Matrix Factorization

TL;DR

This paper introduces a novel column-wise update algorithm for sparse stochastic matrix factorization, ensuring global convergence and near-optimal solutions, with demonstrated effectiveness on synthetic and real datasets.

Contribution

It reformulates sparse stochastic matrix factorization as a nonconvex-nonsmooth problem and provides a convergent algorithm with theoretical guarantees.

Findings

Algorithm converges globally to a critical point.

Generated solutions are nearly global minimizers for each column.

Effective on both synthetic and real data sets.

Abstract

Nonnegative matrix factorization arises widely in machine learning and data analysis. In this paper, for a given factorization of rank r, we consider the sparse stochastic matrix factorization (SSMF) of decomposing a prescribed m-by-n stochastic matrix V into a product of an m-by-r stochastic matrix W and an r-by-n stochastic matrix H, where both W and H are required to be sparse. With the prescribed sparsity level, we reformulate the SSMF as an unconstrained nonconvex-nonsmooth minimization problem and introduce a column-wise update algorithm for solving the minimization problem. We show that our algorithm converges globally. The main advantage of our algorithm is that the generated sequence converges to a special critical point of the cost function, which is nearly a global minimizer over each column vector of the W-factor and is a global minimizer over the H-factor as a whole if there is no sparsity requirement on H. Numerical experiments on both synthetic and real data sets are given to demonstrate the effectiveness of our proposed algorithm.