Closed-form Marginal Likelihood in Gamma-Poisson Matrix Factorization

TL;DR

This paper introduces a closed-form marginal likelihood for the Gamma-Poisson matrix factorization model, providing new insights into parameter estimation and resulting in a robust, automatic model pruning method.

Contribution

It derives a closed-form marginal likelihood for the Gamma-Poisson model, enabling improved estimation and a novel Monte Carlo EM algorithm.

Findings

Robustness of the estimator to over-specified ranks

Automatic pruning of irrelevant dictionary columns

Development of a new Monte Carlo EM algorithm

Abstract

We present novel understandings of the Gamma-Poisson (GaP) model, a probabilistic matrix factorization model for count data. We show that GaP can be rewritten free of the score/activation matrix. This gives us new insights about the estimation of the topic/dictionary matrix by maximum marginal likelihood estimation. In particular, this explains the robustness of this estimator to over-specified values of the factorization rank, especially its ability to automatically prune irrelevant dictionary columns, as empirically observed in previous work. The marginalization of the activation matrix leads in turn to a new Monte Carlo Expectation-Maximization algorithm with favorable properties.