Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems

TL;DR

This paper introduces a novel multi-granular quantization method for embedding large-vocab categorical features in recommender systems, significantly reducing model size while maintaining or improving performance.

Contribution

It proposes the Multi-Granular Quantized Embeddings (MGQE) technique combined with Differentiable Product Quantization (DPQ) to produce compact embeddings, especially for infrequent items.

Findings

Achieves comparable or better recommendation performance with ~20% of original model size.

Demonstrates effectiveness across three recommendation tasks and two datasets.

Addresses overfitting and resource constraints in large-vocab recommender systems.

Abstract

Recommender system models often represent various sparse features like users, items, and categorical features via embeddings. A standard approach is to map each unique feature value to an embedding vector. The size of the produced embedding table grows linearly with the size of the vocabulary. Therefore, a large vocabulary inevitably leads to a gigantic embedding table, creating two severe problems: (i) making model serving intractable in resource-constrained environments; (ii) causing overfitting problems. In this paper, we seek to learn highly compact embeddings for large-vocab sparse features in recommender systems (recsys). First, we show that the novel Differentiable Product Quantization (DPQ) approach can generalize to recsys problems. In addition, to better handle the power-law data distribution commonly seen in recsys, we propose a Multi-Granular Quantized Embeddings (MGQE) technique which learns more compact embeddings for infrequent items. We seek to provide a new angle to improve recommendation performance with compact model sizes. Extensive experiments on three recommendation tasks and two datasets show that we can achieve on par or better performance, with only ~20% of the original model size.