ERASE: Benchmarking Feature Selection Methods for Deep Recommender Systems

TL;DR

This paper introduces ERASE, a comprehensive benchmarking framework for feature selection methods in Deep Recommender Systems, addressing fairness, robustness, and generalizability issues in existing research.

Contribution

ERASE provides a systematic evaluation of eleven feature selection methods across diverse datasets and DRS architectures, filling gaps in fair comparison and practical deployment insights.

Findings

ERASE benchmarks reveal the relative strengths of feature selection methods.

The study identifies robust and stable feature selection techniques for DRS.

Results demonstrate improved recommendation accuracy and efficiency.

Abstract

Deep Recommender Systems (DRS) are increasingly dependent on a large number of feature fields for more precise recommendations. Effective feature selection methods are consequently becoming critical for further enhancing the accuracy and optimizing storage efficiencies to align with the deployment demands. This research area, particularly in the context of DRS, is nascent and faces three core challenges. Firstly, variant experimental setups across research papers often yield unfair comparisons, obscuring practical insights. Secondly, the existing literature's lack of detailed analysis on selection attributes, based on large-scale datasets and a thorough comparison among selection techniques and DRS backbones, restricts the generalizability of findings and impedes deployment on DRS. Lastly, research often focuses on comparing the peak performance achievable by feature selection methods, an approach that is typically computationally infeasible for identifying the optimal hyperparameters and overlooks evaluating the robustness and stability of these methods. To bridge these gaps, this paper presents ERASE, a comprehensive bEnchmaRk for feAture SElection for DRS. ERASE comprises a thorough evaluation of eleven feature selection methods, covering both traditional and deep learning approaches, across four public datasets, private industrial datasets, and a real-world commercial platform, achieving significant enhancement. Our code is available online for ease of reproduction.