Approximation Algorithms for Orthogonal Non-negative Matrix Factorization

TL;DR

This paper introduces the first constant-factor approximation algorithm for orthogonal non-negative matrix factorization (ONMF), demonstrating improved accuracy and strict orthogonality constraints, with applications to clustering tasks.

Contribution

It presents the first constant-factor approximation algorithm for ONMF and establishes a connection to correlation clustering on bipartite graphs.

Findings

Algorithm achieves similar or smaller errors compared to previous methods.

Ensures perfect orthogonality in factorization.

Effective on both synthetic and real-world data.

Abstract

In the non-negative matrix factorization (NMF) problem, the input is an $m\times n$ matrix $M$ with non-negative entries and the goal is to factorize it as $M\approx AW$. The $m\times k$ matrix $A$ and the $k\times n$ matrix $W$ are both constrained to have non-negative entries. This is in contrast to singular value decomposition, where the matrices $A$ and $W$ can have negative entries but must satisfy the orthogonality constraint: the columns of $A$ are orthogonal and the rows of $W$ are also orthogonal. The orthogonal non-negative matrix factorization (ONMF) problem imposes both the non-negativity and the orthogonality constraints, and previous work showed that it leads to better performances than NMF on many clustering tasks. We give the first constant-factor approximation algorithm for ONMF when one or both of $A$ and $W$ are subject to the orthogonality constraint. We also show an interesting connection to the correlation clustering problem on bipartite graphs. Our experiments on synthetic and real-world data show that our algorithm achieves similar or smaller errors compared to previous ONMF algorithms while ensuring perfect orthogonality (many previous algorithms do not satisfy the hard orthogonality constraint).