Leveraging Joint-Diagonalization in Transform-Learning NMF

TL;DR

This paper connects transform-learning NMF to joint-diagonalization, proposing a two-step JD+NMF approach that is effective with many data realizations, but TL-NMF's low-rank constraint remains crucial with limited data.

Contribution

It introduces a novel relation between TL-NMF and joint-diagonalization, and proposes a two-step JD+NMF method, highlighting the importance of TL-NMF's low-rank constraint in data-limited scenarios.

Findings

JD+NMF approximates TL-NMF with large data sets

TL-NMF's low-rank constraint is vital with limited data

Transform learning benefits from joint-diagonalization insights

Abstract

Non-negative matrix factorization with transform learning (TL-NMF) is a recent idea that aims at learning data representations suited to NMF. In this work, we relate TL-NMF to the classical matrix joint-diagonalization (JD) problem. We show that, when the number of data realizations is sufficiently large, TL-NMF can be replaced by a two-step approach -- termed as JD+NMF -- that estimates the transform through JD, prior to NMF computation. In contrast, we found that when the number of data realizations is limited, not only is JD+NMF no longer equivalent to TL-NMF, but the inherent low-rank constraint of TL-NMF turns out to be an essential ingredient to learn meaningful transforms for NMF.