Matryoshka Representation Learning for Recommendation

TL;DR

This paper introduces MRL4Rec, a hierarchical matryoshka representation learning method for recommendation systems that captures user preferences and item features at multiple levels, improving performance over existing methods.

Contribution

It proposes a novel hierarchical representation structure and a triplet construction method, advancing the modeling of user preferences and item features in recommendation systems.

Findings

MRL4Rec outperforms state-of-the-art methods on real datasets.

Hierarchical representations improve recommendation accuracy.

Triplet-specific training guarantees better learning of preferences.

Abstract

Representation learning is essential for deep-neural-network-based recommender systems to capture user preferences and item features within fixed-dimensional user and item vectors. Unlike existing representation learning methods that either treat each user preference and item feature uniformly or categorize them into discrete clusters, we argue that in the real world, user preferences and item features are naturally expressed and organized in a hierarchical manner, leading to a new direction for representation learning. In this paper, we introduce a novel matryoshka representation learning method for recommendation (MRL4Rec), by which we restructure user and item vectors into matryoshka representations with incrementally dimensional and overlapping vector spaces to explicitly represent user preferences and item features at different hierarchical levels. We theoretically establish that constructing training triplets specific to each level is pivotal in guaranteeing accurate matryoshka representation learning. Subsequently, we propose the matryoshka negative sampling mechanism to construct training triplets, which further ensures the effectiveness of the matryoshka representation learning in capturing hierarchical user preferences and item features. The experiments demonstrate that MRL4Rec can consistently and substantially outperform a number of state-of-the-art competitors on several real-life datasets. Our code is publicly available at https://github.com/Riwei-HEU/MRL.