Stochastic variance reduced multiplicative update for nonnegative matrix factorization

TL;DR

This paper introduces a variance-reduced stochastic multiplicative update method for nonnegative matrix factorization, significantly improving convergence speed and outperforming existing algorithms on various datasets.

Contribution

It presents a novel variance reduction technique for stochastic MU in NMF, enhancing convergence and robustness over prior methods.

Findings

Outperforms state-of-the-art algorithms on synthetic datasets

Demonstrates robustness and faster convergence on real-world data

Provides a scalable solution for large-scale NMF problems

Abstract

Nonnegative matrix factorization (NMF), a dimensionality reduction and factor analysis method, is a special case in which factor matrices have low-rank nonnegative constraints. Considering the stochastic learning in NMF, we specifically address the multiplicative update (MU) rule, which is the most popular, but which has slow convergence property. This present paper introduces on the stochastic MU rule a variance-reduced technique of stochastic gradient. Numerical comparisons suggest that our proposed algorithms robustly outperform state-of-the-art algorithms across different synthetic and real-world datasets.