The Diagonalized Newton Algorithm for Nonnegative Matrix Factorization

TL;DR

This paper introduces the Diagonalized Newton Algorithm (DNA) for non-negative matrix factorization, achieving faster convergence than traditional methods while maintaining simplicity, especially effective for high-rank problems on modern hardware.

Contribution

The paper proposes a novel diagonalized Newton algorithm that improves convergence speed in NMF without sacrificing simplicity, suitable for high-rank matrices.

Findings

DNA outperforms multiplicative updates in convergence speed

The algorithm is effective on various real-world datasets

It is computationally efficient on modern hardware

Abstract

Non-negative matrix factorization (NMF) has become a popular machine learning approach to many problems in text mining, speech and image processing, bio-informatics and seismic data analysis to name a few. In NMF, a matrix of non-negative data is approximated by the low-rank product of two matrices with non-negative entries. In this paper, the approximation quality is measured by the Kullback-Leibler divergence between the data and its low-rank reconstruction. The existence of the simple multiplicative update (MU) algorithm for computing the matrix factors has contributed to the success of NMF. Despite the availability of algorithms showing faster convergence, MU remains popular due to its simplicity. In this paper, a diagonalized Newton algorithm (DNA) is proposed showing faster convergence while the implementation remains simple and suitable for high-rank problems. The DNA algorithm is applied to various publicly available data sets, showing a substantial speed-up on modern hardware.