Enhancing Scalability in Recommender Systems through Lottery Ticket Hypothesis and Knowledge Distillation-based Neural Network Pruning

TL;DR

This paper proposes a novel neural network pruning method combining the Lottery Ticket Hypothesis and Knowledge Distillation to improve scalability and efficiency of recommender systems on edge devices, reducing power consumption significantly.

Contribution

It introduces a new pruning approach integrating LTH and KD, specifically addressing scalability issues in recommender systems with practical efficiency gains.

Findings

GPU power consumption reduced by up to 66.67%

Effective model size reduction without accuracy loss

First application of LTH and KD in recommender system pruning

Abstract

This study introduces an innovative approach aimed at the efficient pruning of neural networks, with a particular focus on their deployment on edge devices. Our method involves the integration of the Lottery Ticket Hypothesis (LTH) with the Knowledge Distillation (KD) framework, resulting in the formulation of three distinct pruning models. These models have been developed to address scalability issue in recommender systems, whereby the complexities of deep learning models have hindered their practical deployment. With judicious application of the pruning techniques, we effectively curtail the power consumption and model dimensions without compromising on accuracy. Empirical evaluation has been performed using two real world datasets from diverse domains against two baselines. Gratifyingly, our approaches yielded a GPU computation-power reduction of up to 66.67%. Notably, our study contributes to the field of recommendation system by pioneering the application of LTH and KD.