On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective

TL;DR

This paper introduces Shaver, a novel embedding pruning method for content-based recommenders that uses Shapley values from cooperative game theory to efficiently reduce model size without retraining, suitable for resource-limited devices.

Contribution

We propose Shaver, a contribution-based embedding pruning method using Shapley values, with an efficient estimation technique and a field-aware codebook to improve on existing methods.

Findings

Shaver achieves competitive recommendation accuracy across various parameter budgets.

The proposed method significantly reduces computation costs compared to traditional Shapley value calculations.

Experiments on three real-world datasets validate the effectiveness of Shaver in resource-constrained environments.

Abstract

Content-based Recommender Systems (CRSs) play a crucial role in shaping user experiences in e-commerce, online advertising, and personalized recommendations. However, due to the vast amount of categorical features, the embedding tables used in CRS models pose a significant storage bottleneck for real-world deployment, especially on resource-constrained devices. To address this problem, various embedding pruning methods have been proposed, but most existing ones require expensive retraining steps for each target parameter budget, leading to enormous computation costs. In reality, this computation cost is a major hurdle in real-world applications with diverse storage requirements, such as federated learning and streaming settings. In this paper, we propose Shapley Value-guided Embedding Reduction (Shaver) as our response. With Shaver, we view the problem from a cooperative game perspective, and quantify each embedding parameter's contribution with Shapley values to facilitate contribution-based parameter pruning. To address the inherently high computation costs of Shapley values, we propose an efficient and unbiased method to estimate Shapley values of a CRS's embedding parameters. Moreover, in the pruning stage, we put forward a field-aware codebook to mitigate the information loss in the traditional zero-out treatment. Through extensive experiments on three real-world datasets, Shaver has demonstrated competitive performance with lightweight recommendation models across various parameter budgets. The source code is available at https://github.com/chenxing1999/shaver