Using Underapproximations for Sparse Nonnegative Matrix Factorization

TL;DR

This paper introduces a novel underapproximation method for sparse nonnegative matrix factorization (NMF) using Lagrangian relaxation, demonstrating its effectiveness in producing sparse, low-error representations across various datasets.

Contribution

The paper presents a new underapproximation approach for NMF that leverages Lagrangian relaxation and recursive solving, with proven NP-hardness and competitive empirical results.

Findings

Produces sparse, low-error representations

Comparable or superior to standard NMF techniques

Effective across multiple image datasets

Abstract

Nonnegative Matrix Factorization consists in (approximately) factorizing a nonnegative data matrix by the product of two low-rank nonnegative matrices. It has been successfully applied as a data analysis technique in numerous domains, e.g., text mining, image processing, microarray data analysis, collaborative filtering, etc. We introduce a novel approach to solve NMF problems, based on the use of an underapproximation technique, and show its effectiveness to obtain sparse solutions. This approach, based on Lagrangian relaxation, allows the resolution of NMF problems in a recursive fashion. We also prove that the underapproximation problem is NP-hard for any fixed factorization rank, using a reduction of the maximum edge biclique problem in bipartite graphs. We test two variants of our underapproximation approach on several standard image datasets and show that they provide sparse part-based representations with low reconstruction error. Our results are comparable and sometimes superior to those obtained by two standard Sparse Nonnegative Matrix Factorization techniques.