Novel Algorithms based on Majorization Minimization for Nonnegative Matrix Factorization

TL;DR

This paper introduces two new algorithms for nonnegative matrix factorization based on Majorization Minimization, offering improved speed and convergence, with proven convergence to stationary points and applications in big data.

Contribution

The paper proposes two novel MM-based algorithms for NMF, with different update schemes, and demonstrates their superior performance over existing methods.

Findings

Algorithms converge to stationary points.

Proposed methods outperform existing algorithms in speed.

Effective for large-scale data applications.

Abstract

Matrix decomposition is ubiquitous and has applications in various fields like speech processing, data mining and image processing to name a few. Under matrix decomposition, nonnegative matrix factorization is used to decompose a nonnegative matrix into a product of two nonnegative matrices which gives some meaningful interpretation of the data. Thus, nonnegative matrix factorization has an edge over the other decomposition techniques. In this paper, we propose two novel iterative algorithms based on Majorization Minimization (MM)-in which we formulate a novel upper bound and minimize it to get a closed form solution at every iteration. Since the algorithms are based on MM, it is ensured that the proposed methods will be monotonic. The proposed algorithms differ in the updating approach of the two nonnegative matrices. The first algorithm-Iterative Nonnegative Matrix Factorization (INOM) sequentially updates the two nonnegative matrices while the second algorithm-Parallel Iterative Nonnegative Matrix Factorization (PARINOM) parallely updates them. We also prove that the proposed algorithms converge to the stationary point of the problem. Simulations were conducted to compare the proposed methods with the existing ones and was found that the proposed algorithms performs better than the existing ones in terms of computational speed and convergence. KeyWords: Nonnegative matrix factorization, Majorization Minimization, Big Data, Parallel, Multiplicative Update