Probabilistic semi-nonnegative matrix factorization: a Skellam-based framework

TL;DR

This paper introduces Skellam-SNMF, a probabilistic framework for semi-nonnegative matrix factorization utilizing Skellam distributions, with algorithms for Bayesian inference and a new divergence measure, demonstrating improved clustering performance.

Contribution

It proposes a novel Skellam-based probabilistic model for SNMF, along with inference algorithms and a new divergence, advancing the state-of-the-art in matrix factorization methods.

Findings

Outperforms classic SNMF in clustering tasks

Provides Bayesian inference algorithms for SNMF

Introduces a new divergence measure for real-valued data

Abstract

We present a new probabilistic model to address semi-nonnegative matrix factorization (SNMF), called Skellam-SNMF. It is a hierarchical generative model consisting of prior components, Skellam-distributed hidden variables and observed data. Two inference algorithms are derived: Expectation-Maximization (EM) algorithm for maximum \emph{a posteriori} estimation and Variational Bayes EM (VBEM) for full Bayesian inference, including the estimation of parameters prior distribution. From this Skellam-based model, we also introduce a new divergence $\mathcal{D}$ between a real-valued target data $x$ and two nonnegative parameters $\lambda_{0}$ and $\lambda_{1}$ such that $\mathcal{D}\left(x\mid\lambda_{0},\lambda_{1}\right)=0\Leftrightarrow x=\lambda_{0}-\lambda_{1}$, which is a generalization of the Kullback-Leibler (KL) divergence. Finally, we conduct experimental studies on those new algorithms in order to understand their behavior and prove that they can outperform the classic SNMF approach on real data in a task of automatic clustering.