Differentiable Neural Input Search for Recommender Systems

TL;DR

This paper introduces DNIS, a differentiable method for optimizing feature embedding dimensions in recommender systems, leading to better performance and efficiency by allowing flexible, continuous dimension search.

Contribution

We propose a model-agnostic, differentiable input search method that improves embedding dimension selection in latent factor models for recommendation tasks.

Findings

DNIS outperforms existing neural input search methods in predictive accuracy.

It achieves this with fewer embedding parameters.

It requires less computational time.

Abstract

Latent factor models are the driving forces of the state-of-the-art recommender systems, with an important insight of vectorizing raw input features into dense embeddings. The dimensions of different feature embeddings are often set to a same value empirically, which limits the predictive performance of latent factor models. Existing works have proposed heuristic or reinforcement learning-based methods to search for mixed feature embedding dimensions. For efficiency concern, these methods typically choose embedding dimensions from a restricted set of candidate dimensions. However, this restriction will hurt the flexibility of dimension selection, leading to suboptimal performance of search results. In this paper, we propose Differentiable Neural Input Search (DNIS), a method that searches for mixed feature embedding dimensions in a more flexible space through continuous relaxation and differentiable optimization. The key idea is to introduce a soft selection layer that controls the significance of each embedding dimension, and optimize this layer according to model's validation performance. DNIS is model-agnostic and thus can be seamlessly incorporated with existing latent factor models for recommendation. We conduct experiments with various architectures of latent factor models on three public real-world datasets for rating prediction, Click-Through-Rate (CTR) prediction, and top-k item recommendation. The results demonstrate that our method achieves the best predictive performance compared with existing neural input search approaches with fewer embedding parameters and less time cost.