Sum-of-norms regularized Nonnegative Matrix Factorization

TL;DR

This paper introduces SON-NMF, a novel method that automatically estimates the nonnegative rank during NMF, effectively handling rank-deficient data and spectral variability, with a new low-cost algorithm for practical application.

Contribution

The paper proposes SON-NMF, a new approach that estimates the nonnegative rank on-the-fly using sum-of-norm regularization, and develops an efficient first-order algorithm for its solution.

Findings

SON-NMF accurately estimates the nonnegative rank without prior knowledge.

It effectively handles rank-deficient data matrices and weak components.

The proposed algorithm has low per-iteration computational cost.

Abstract

When applying nonnegative matrix factorization (NMF), the rank parameter is generally unknown. This rank, called the nonnegative rank, is usually estimated heuristically since computing its exact value is NP-hard. In this work, we propose an approximation method to estimate the rank on-the-fly while solving NMF. We use the sum-of-norm (SON), a group-lasso structure that encourages pairwise similarity, to reduce the rank of a factor matrix when the initial rank is overestimated. On various datasets, SON-NMF can reveal the correct nonnegative rank of the data without prior knowledge or parameter tuning. SON-NMF is a nonconvex, nonsmooth, non-separable, and non-proximable problem, making it nontrivial to solve. First, since rank estimation in NMF is NP-hard, the proposed approach does not benefit from lower computational complexity. Using a graph-theoretic argument, we prove that the complexity of SON-NMF is essentially irreducible. Second, the per-iteration cost of algorithms for SON-NMF can be high. This motivates us to propose a first-order BCD algorithm that approximately solves SON-NMF with low per-iteration cost via the proximal average operator. SON-NMF exhibits favorable features for applications. Besides the ability to automatically estimate the rank from data, SON-NMF can handle rank-deficient data matrices and detect weak components with small energy. Furthermore, in hyperspectral imaging, SON-NMF naturally addresses the issue of spectral variability.