Embedding Compression in Recommender Systems: A Survey

TL;DR

This survey reviews various embedding compression techniques in recommender systems, aiming to reduce memory usage and improve efficiency by categorizing approaches into low-precision, mixed-dimension, and weight-sharing methods.

Contribution

It provides a comprehensive classification and analysis of existing embedding compression methods in recommender systems, highlighting future research directions.

Findings

Embedding compression reduces memory costs in recommender systems.

Three main categories of compression techniques are identified.

Future prospects include developing more efficient and scalable methods.

Abstract

To alleviate the problem of information explosion, recommender systems are widely deployed to provide personalized information filtering services. Usually, embedding tables are employed in recommender systems to transform high-dimensional sparse one-hot vectors into dense real-valued embeddings. However, the embedding tables are huge and account for most of the parameters in industrial-scale recommender systems. In order to reduce memory costs and improve efficiency, various approaches are proposed to compress the embedding tables. In this survey, we provide a comprehensive review of embedding compression approaches in recommender systems. We first introduce deep learning recommendation models and the basic concept of embedding compression in recommender systems. Subsequently, we systematically organize existing approaches into three categories, namely low-precision, mixed-dimension, and weight-sharing, respectively. Lastly, we summarize the survey with some general suggestions and provide future prospects for this field.