Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation

TL;DR

This paper compares various inference methods for Bayesian nonnegative matrix factorisation and tri-factorisation, highlighting the advantages of the variational approach and Bayesian automatic relevance determination in terms of convergence, robustness, and model selection.

Contribution

Introduces the variational Bayesian inference method for Bayesian nonnegative models and evaluates its performance against other inference techniques.

Findings

Variational Bayesian inference shows competitive convergence and robustness.

Bayesian automatic relevance determination enables automatic model selection.

The methods perform well on both synthetic and real-world datasets.

Abstract

In this paper, we study the trade-offs of different inference approaches for Bayesian matrix factorisation methods, which are commonly used for predicting missing values, and for finding patterns in the data. In particular, we consider Bayesian nonnegative variants of matrix factorisation and tri-factorisation, and compare non-probabilistic inference, Gibbs sampling, variational Bayesian inference, and a maximum-a-posteriori approach. The variational approach is new for the Bayesian nonnegative models. We compare their convergence, and robustness to noise and sparsity of the data, on both synthetic and real-world datasets. Furthermore, we extend the models with the Bayesian automatic relevance determination prior, allowing the models to perform automatic model selection, and demonstrate its efficiency.