Collaborative Self-Attention for Recommender Systems

TL;DR

This paper introduces a novel collaborative self-attention mechanism for inductive matrix completion in recommender systems, enhancing the expressiveness of feature representations and improving recommendation accuracy.

Contribution

It proposes the CSA model that generalizes self-attention to learn context-aware feature representations for better matrix completion in RS.

Findings

CSA outperforms existing IMC methods on large-scale datasets

The model effectively captures feature interactions in recommender systems

Experimental results show significant accuracy improvements

Abstract

Recommender systems (RS), which have been an essential part in a wide range of applications, can be formulated as a matrix completion (MC) problem. To boost the performance of MC, matrix completion with side information, called inductive matrix completion (IMC), was further proposed. In real applications, the factorized version of IMC is more favored due to its efficiency of optimization and implementation. Regarding the factorized version, traditional IMC method can be interpreted as learning an individual representation for each feature, which is independent from each other. Moreover, representations for the same features are shared across all users/items. However, the independent characteristic for features and shared characteristic for the same features across all users/items may limit the expressiveness of the model. The limitation also exists in variants of IMC, such as deep learning based IMC models. To break the limitation, we generalize recent advances of self-attention mechanism to IMC and propose a context-aware model called collaborative self-attention (CSA), which can jointly learn context-aware representations for features and perform inductive matrix completion process. Extensive experiments on three large-scale datasets from real RS applications demonstrate effectiveness of CSA.