Clustering and Latent Semantic Indexing Aspects of the Nonnegative Matrix Factorization

TL;DR

This paper provides a theoretical foundation for the clustering capabilities of nonnegative matrix factorization (NMF) by linking it to graph clustering and evaluates its latent semantic indexing performance against SVD.

Contribution

It establishes a theoretical equivalence between NMF and graph clustering using KKT conditions and evaluates NMF's LSI ability compared to SVD.

Findings

NMF objective is equivalent to graph clustering objective.

NMF effectively addresses synonymy and polysemy in synthetic datasets.

NMF's LSI performance is comparable or superior to SVD in experiments.

Abstract

This paper provides a theoretical support for clustering aspect of the nonnegative matrix factorization (NMF). By utilizing the Karush-Kuhn-Tucker optimality conditions, we show that NMF objective is equivalent to graph clustering objective, so clustering aspect of the NMF has a solid justification. Different from previous approaches which usually discard the nonnegativity constraints, our approach guarantees the stationary point being used in deriving the equivalence is located on the feasible region in the nonnegative orthant. Additionally, since clustering capability of a matrix decomposition technique can sometimes imply its latent semantic indexing (LSI) aspect, we will also evaluate LSI aspect of the NMF by showing its capability in solving the synonymy and polysemy problems in synthetic datasets. And more extensive evaluation will be conducted by comparing LSI performances of the NMF and the singular value decomposition (SVD), the standard LSI method, using some standard datasets.