Co-Separable Nonnegative Matrix Factorization

TL;DR

This paper introduces Co-Separable Nonnegative Matrix Factorization (CoS-NMF), a generalized model that relaxes the separability assumption in NMF, with proven mathematical properties and effective algorithms, improving data approximation and clustering.

Contribution

It generalizes the separability assumption in NMF to a co-separable model, providing new mathematical insights and an efficient optimization algorithm.

Findings

CoS-NMF effectively approximates data matrices in experiments.

It outperforms state-of-the-art methods in co-clustering tasks.

The model maintains good data reconstruction quality.

Abstract

Nonnegative matrix factorization (NMF) is a popular model in the field of pattern recognition. It aims to find a low rank approximation for nonnegative data M by a product of two nonnegative matrices W and H. In general, NMF is NP-hard to solve while it can be solved efficiently under separability assumption, which requires the columns of factor matrix are equal to columns of the input matrix. In this paper, we generalize separability assumption based on 3-factor NMF M=P_1SP_2, and require that S is a sub-matrix of the input matrix. We refer to this NMF as a Co-Separable NMF (CoS-NMF). We discuss some mathematics properties of CoS-NMF, and present the relationships with other related matrix factorizations such as CUR decomposition, generalized separable NMF(GS-NMF), and bi-orthogonal tri-factorization (BiOR-NM3F). An optimization model for CoS-NMF is proposed and alternated fast gradient method is employed to solve the model. Numerical experiments on synthetic datasets, document datasets and facial databases are conducted to verify the effectiveness of our CoS-NMF model. Compared to state-of-the-art methods, CoS-NMF model performs very well in co-clustering task, and preserves a good approximation to the input data matrix as well.