Meta-Wrapper: Differentiable Wrapping Operator for User Interest Selection in CTR Prediction

TL;DR

This paper introduces Meta-Wrapper, a differentiable feature selection method for user interest modeling in CTR prediction, utilizing bilevel optimization and meta-learning to improve accuracy and robustness.

Contribution

It proposes a novel differentiable wrapper method with bilevel optimization and meta-learning for user interest selection in CTR prediction, enhancing feature selection efficiency and overfitting resistance.

Findings

Outperforms existing methods on three public datasets.

Improves CTR prediction accuracy.

Provides theoretical convergence and overfitting resistance analysis.

Abstract

Click-through rate (CTR) prediction, whose goal is to predict the probability of the user to click on an item, has become increasingly significant in the recommender systems. Recently, some deep learning models with the ability to automatically extract the user interest from his/her behaviors have achieved great success. In these work, the attention mechanism is used to select the user interested items in historical behaviors, improving the performance of the CTR predictor. Normally, these attentive modules can be jointly trained with the base predictor by using gradient descents. In this paper, we regard user interest modeling as a feature selection problem, which we call user interest selection. For such a problem, we propose a novel approach under the framework of the wrapper method, which is named Meta-Wrapper. More specifically, we use a differentiable module as our wrapping operator and then recast its learning problem as a continuous bilevel optimization. Moreover, we use a meta-learning algorithm to solve the optimization and theoretically prove its convergence. Meanwhile, we also provide theoretical analysis to show that our proposed method 1) efficiencies the wrapper-based feature selection, and 2) achieves better resistance to overfitting. Finally, extensive experiments on three public datasets manifest the superiority of our method in boosting the performance of CTR prediction.