Orthogonal Nonnegative Matrix Factorization with the Kullback-Leibler divergence

TL;DR

This paper introduces a novel orthogonal nonnegative matrix factorization method that minimizes the Kullback-Leibler divergence, improving modeling of sparse count data in clustering applications.

Contribution

It proposes a new KL divergence-based ONMF model and algorithm, addressing limitations of the Frobenius norm approach for certain data types.

Findings

KL-ONMF outperforms Frobenius-norm ONMF in document classification.

The method is effective for hyperspectral image unmixing.

It better models sparse count data in various applications.

Abstract

Orthogonal nonnegative matrix factorization (ONMF) has become a standard approach for clustering. As far as we know, most works on ONMF rely on the Frobenius norm to assess the quality of the approximation. This paper presents a new model and algorithm for ONMF that minimizes the Kullback-Leibler (KL) divergence. As opposed to the Frobenius norm which assumes Gaussian noise, the KL divergence is the maximum likelihood estimator for Poisson-distributed data, which can model better sparse vectors of word counts in document data sets and photo counting processes in imaging. We develop an algorithm based on alternating optimization, KL-ONMF, and show that it performs favorably with the Frobenius-norm based ONMF for document classification and hyperspectral image unmixing.