Course Project Report: Comparing MCMC and Variational Inference for Bayesian Probabilistic Matrix Factorization on the MovieLens Dataset

TL;DR

This paper compares MCMC and Variational Inference for Bayesian Probabilistic Matrix Factorization on MovieLens, highlighting their differences in convergence speed, accuracy, and efficiency.

Contribution

It provides a comparative analysis of MCMC and VI methods for Bayesian matrix factorization in recommendation systems, with empirical evaluation on MovieLens.

Findings

VI converges faster than MCMC

MCMC yields more accurate posterior estimates

VI is more computationally efficient

Abstract

This is a course project report with complete methodology, experiments, references and mathematical derivations. Matrix factorization [1] is a widely used technique in recommendation systems. Probabilistic Matrix Factorization (PMF) [2] extends traditional matrix factorization by incorporating probability distributions over latent factors, allowing for uncertainty quantification. However, computing the posterior distribution is intractable due to the high-dimensional integral. To address this, we employ two Bayesian inference methods: Markov Chain Monte Carlo (MCMC) [3, 4] and Variational Inference (VI) [5, 6] to approximate the posterior. We evaluate their performance on MovieLens dataset [7] and compare their convergence speed, predictive accuracy, and computational efficiency. Experimental results demonstrate that VI offers faster convergence, while MCMC provides more accurate posterior estimates.