MM-algorithms for traditional and convex NMF with Tweedie and Negative Binomial cost functions and empirical evaluation

TL;DR

This paper introduces a unified framework for non-negative matrix factorization under various distributional assumptions, including Negative Binomial and Tweedie models, with new algorithms and empirical evaluation demonstrating their effectiveness.

Contribution

It develops a Majorize-Minimisation based approach for convex and traditional NMF with novel updates for complex noise models, and provides the first implementations of several convex NMF models.

Findings

Choice of noise model significantly impacts model fit.

Convex NMF offers an efficient alternative in large class scenarios.

Empirical results on mutational and word count data validate the approach.

Abstract

Non-negative matrix factorisation (NMF) is a widely used tool for unsupervised learning and feature extraction, with applications ranging from genomics to text analysis and signal processing. Standard formulations of NMF are typically derived under Gaussian or Poisson noise assumptions, which may be inadequate for data exhibiting overdispersion or other complex mean-variance relationships. In this paper, we develop a unified framework for both traditional and convex NMF under a broad class of distributional assumptions, including Negative Binomial and Tweedie models, where the connection between the Tweedie and the $\beta$-divergence is also highlighted. Using a Majorize-Minimisation approach, we derive multiplicative update rules for all considered models, and novel updates for convex NMF with Poisson and Negative Binomial cost functions. We provide a unified implementation of all considered models, including the first implementations of several convex NMF models. Empirical evaluations on mutational and word count data demonstrate that the choice of noise model critically affects model fit and feature recovery, and that convex NMF can provide an efficient and robust alternative to traditional NMF in scenarios where the number of classes is large. The code for our proposed updates is available in the R package nmfgenr and can be found at https://github.com/MartaPelizzola/nmfgenr.