Factoring nonnegative matrices with linear programs

TL;DR

This paper introduces a linear programming-based method for nonnegative matrix factorization that identifies salient features in data, offering theoretical guarantees, scalability, and superior practical performance on large datasets.

Contribution

The paper presents a novel linear programming approach for NMF that extends to general noise models and is more scalable than previous methods.

Findings

The method provides theoretical guarantees similar to existing algorithms.

It scales efficiently to large, multigigabyte matrices.

Experiments show superior performance on synthetic and real data.

Abstract

This paper describes a new approach, based on linear programming, for computing nonnegative matrix factorizations (NMFs). The key idea is a data-driven model for the factorization where the most salient features in the data are used to express the remaining features. More precisely, given a data matrix X, the algorithm identifies a matrix C such that X approximately equals CX and some linear constraints. The constraints are chosen to ensure that the matrix C selects features; these features can then be used to find a low-rank NMF of X. A theoretical analysis demonstrates that this approach has guarantees similar to those of the recent NMF algorithm of Arora et al. (2012). In contrast with this earlier work, the proposed method extends to more general noise models and leads to efficient, scalable algorithms. Experiments with synthetic and real datasets provide evidence that the new approach is also superior in practice. An optimized C++ implementation can factor a multigigabyte matrix in a matter of minutes.