An optimal pairwise merge algorithm improves the quality and consistency of nonnegative matrix factorization

TL;DR

This paper introduces a pairwise merge algorithm for NMF that enhances solution quality and consistency by escaping poor local minima and is compatible with various NMF methods, with minimal additional computational cost.

Contribution

The paper presents an analytically-solvable pairwise merge strategy in higher-dimensional space to improve NMF solutions and solution consistency.

Findings

Helps NMF solutions escape poor local minima.

Achieves greater consistency of NMF solutions.

Maintains similar computational performance to existing methods.

Abstract

Non-negative matrix factorization (NMF) is a key technique for feature extraction and widely used in source separation. However, existing algorithms may converge to poor local minima, or to one of several minima with similar objective value but differing feature parametrizations. Here we show that some of these weaknesses may be mitigated by performing NMF in a higher-dimensional feature space and then iteratively combining components with an analytically-solvable pairwise merge strategy. Experimental results demonstrate our method helps non-ideal NMF solutions escape to better local optima and achieve greater consistency of the solutions. Despite these extra steps, our approach exhibits similar computational performance to established methods by reducing the occurrence of "plateau phenomenon" near saddle points. Moreover, the results also illustrate that our method is compatible with different NMF algorithms. Thus, this can be recommended as a preferred approach for most applications of NMF.