Clustered Embedding Learning for Recommender Systems

TL;DR

This paper introduces Clustered Embedding Learning (CEL), a novel framework that improves recommender system performance for cold users and items while significantly reducing memory costs by dynamically clustering entities.

Contribution

CEL is a flexible, plug-and-play embedding method that enables automatic clustering, theoretical guarantees on cluster number, and efficient online updates, addressing key limitations of existing approaches.

Findings

CEL improves AUC by 0.6% in business applications.

Embedding table size reduced by 2650 times.

Consistently outperforms state-of-the-art methods on multiple datasets.

Abstract

In recent years, recommender systems have advanced rapidly, where embedding learning for users and items plays a critical role. A standard method learns a unique embedding vector for each user and item. However, such a method has two important limitations in real-world applications: 1) it is hard to learn embeddings that generalize well for users and items with rare interactions on their own; and 2) it may incur unbearably high memory costs when the number of users and items scales up. Existing approaches either can only address one of the limitations or have flawed overall performances. In this paper, we propose Clustered Embedding Learning (CEL) as an integrated solution to these two problems. CEL is a plug-and-play embedding learning framework that can be combined with any differentiable feature interaction model. It is capable of achieving improved performance, especially for cold users and items, with reduced memory cost. CEL enables automatic and dynamic clustering of users and items in a top-down fashion, where clustered entities jointly learn a shared embedding. The accelerated version of CEL has an optimal time complexity, which supports efficient online updates. Theoretically, we prove the identifiability and the existence of a unique optimal number of clusters for CEL in the context of nonnegative matrix factorization. Empirically, we validate the effectiveness of CEL on three public datasets and one business dataset, showing its consistently superior performance against current state-of-the-art methods. In particular, when incorporating CEL into the business model, it brings an improvement of $+0.6\%$ in AUC, which translates into a significant revenue gain; meanwhile, the size of the embedding table gets $2650$ times smaller.