Towards a Better Understanding of Linear Models for Recommendation

TL;DR

This paper provides a theoretical analysis of linear models for recommendation, revealing their relationship with matrix factorization, and introduces a new algorithm to explore related models, demonstrating competitive performance with state-of-the-art methods.

Contribution

It offers a theoretical connection between linear regression and matrix factorization models, and proposes a new hyperparameter search algorithm for these models.

Findings

Linear models are inherently related but differ in scaling singular values.

Closed-form solutions are competitive with state-of-the-art models.

The new algorithm effectively explores nearby models and hyperparameters.

Abstract

Recently, linear regression models, such as EASE and SLIM, have shown to often produce rather competitive results against more sophisticated deep learning models. On the other side, the (weighted) matrix factorization approaches have been popular choices for recommendation in the past and widely adopted in the industry. In this work, we aim to theoretically understand the relationship between these two approaches, which are the cornerstones of model-based recommendations. Through the derivation and analysis of the closed-form solutions for two basic regression and matrix factorization approaches, we found these two approaches are indeed inherently related but also diverge in how they "scale-down" the singular values of the original user-item interaction matrix. This analysis also helps resolve the questions related to the regularization parameter range and model complexities. We further introduce a new learning algorithm in searching (hyper)parameters for the closed-form solution and utilize it to discover the nearby models of the existing solutions. The experimental results demonstrate that the basic models and their closed-form solutions are indeed quite competitive against the state-of-the-art models, thus, confirming the validity of studying the basic models. The effectiveness of exploring the nearby models are also experimentally validated.