Variational Factorization Machines for Preference Elicitation in Large-Scale Recommender Systems

TL;DR

This paper introduces a variational approach to factorization machines that enables scalable training and provides confidence intervals, improving large-scale recommendation systems and active learning applications.

Contribution

It proposes a variational formulation of FMs optimized with stochastic gradient descent, allowing efficient large-scale learning with confidence estimation.

Findings

Comparable or better prediction accuracy than existing methods

Effective in active learning and preference elicitation

Scalable to large datasets

Abstract

Factorization machines (FMs) are a powerful tool for regression and classification in the context of sparse observations, that has been successfully applied to collaborative filtering, especially when side information over users or items is available. Bayesian formulations of FMs have been proposed to provide confidence intervals over the predictions made by the model, however they usually involve Markov-chain Monte Carlo methods that require many samples to provide accurate predictions, resulting in slow training in the context of large-scale data. In this paper, we propose a variational formulation of factorization machines that allows us to derive a simple objective that can be easily optimized using standard mini-batch stochastic gradient descent, making it amenable to large-scale data. Our algorithm learns an approximate posterior distribution over the user and item parameters, which leads to confidence intervals over the predictions. We show, using several datasets, that it has comparable or better performance than existing methods in terms of prediction accuracy, and provide some applications in active learning strategies, e.g., preference elicitation techniques.