Factorizing LambdaMART for cold start recommendations

TL;DR

This paper introduces LambdaMART-MF, a novel low-rank matrix factorization approach using gradient boosted trees for improved cold start recommendations, incorporating regularization and a weighted NDCG variant.

Contribution

It proposes LambdaMART-MF, a new algorithm that factorizes LambdaMART with low-rank representations and regularizers, enhancing recommendation accuracy especially in cold start scenarios.

Findings

Outperforms state-of-the-art algorithms on two datasets

Effective in cold start and matrix completion settings

Regularization improves model generalization

Abstract

Recommendation systems often rely on point-wise loss metrics such as the mean squared error. However, in real recommendation settings only few items are presented to a user. This observation has recently encouraged the use of rank-based metrics. LambdaMART is the state-of-the-art algorithm in learning to rank which relies on such a metric. Despite its success it does not have a principled regularization mechanism relying in empirical approaches to control model complexity leaving it thus prone to overfitting. Motivated by the fact that very often the users' and items' descriptions as well as the preference behavior can be well summarized by a small number of hidden factors, we propose a novel algorithm, LambdaMART Matrix Factorization (LambdaMART-MF), that learns a low rank latent representation of users and items using gradient boosted trees. The algorithm factorizes lambdaMART by defining relevance scores as the inner product of the learned representations of the users and items. The low rank is essentially a model complexity controller; on top of it we propose additional regularizers to constraint the learned latent representations that reflect the user and item manifolds as these are defined by their original feature based descriptors and the preference behavior. Finally we also propose to use a weighted variant of NDCG to reduce the penalty for similar items with large rating discrepancy. We experiment on two very different recommendation datasets, meta-mining and movies-users, and evaluate the performance of LambdaMART-MF, with and without regularization, in the cold start setting as well as in the simpler matrix completion setting. In both cases it outperforms in a significant manner current state of the art algorithms.