iHAS: Instance-wise Hierarchical Architecture Search for Deep Learning Recommendation Models

TL;DR

This paper introduces iHAS, an innovative framework that automates neural architecture search at the instance level for recommender systems, optimizing embedding dimensions and improving performance by considering item and user heterogeneity.

Contribution

iHAS is the first to perform instance-wise hierarchical architecture search, incorporating clustering and retraining to tailor models to different sample groups in recommender systems.

Findings

iHAS outperforms baseline models on benchmark datasets.

The framework demonstrates strong transferability across various deep recommendation models.

Experimental results show significant improvements in recommendation accuracy.

Abstract

Current recommender systems employ large-sized embedding tables with uniform dimensions for all features, leading to overfitting, high computational cost, and suboptimal generalizing performance. Many techniques aim to solve this issue by feature selection or embedding dimension search. However, these techniques typically select a fixed subset of features or embedding dimensions for all instances and feed all instances into one recommender model without considering heterogeneity between items or users. This paper proposes a novel instance-wise Hierarchical Architecture Search framework, iHAS, which automates neural architecture search at the instance level. Specifically, iHAS incorporates three stages: searching, clustering, and retraining. The searching stage identifies optimal instance-wise embedding dimensions across different field features via carefully designed Bernoulli gates with stochastic selection and regularizers. After obtaining these dimensions, the clustering stage divides samples into distinct groups via a deterministic selection approach of Bernoulli gates. The retraining stage then constructs different recommender models, each one designed with optimal dimensions for the corresponding group. We conduct extensive experiments to evaluate the proposed iHAS on two public benchmark datasets from a real-world recommender system. The experimental results demonstrate the effectiveness of iHAS and its outstanding transferability to widely-used deep recommendation models.