From Feature Interaction to Feature Generation: A Generative Paradigm of CTR Prediction Models

TL;DR

This paper introduces a supervised generative framework for CTR prediction that replaces traditional feature interaction methods, effectively reducing redundancy and improving performance across models.

Contribution

It proposes the Supervised Feature Generation (SFG) framework, a novel paradigm shift from discriminative to generative modeling in CTR prediction, enhancing model robustness and accuracy.

Findings

SFG reduces embedding collapse and redundancy.

SFG improves CTR prediction accuracy across datasets.

Framework is compatible with existing models.

Abstract

Click-Through Rate (CTR) prediction, a core task in recommendation systems, aims to estimate the probability of users clicking on items. Existing models predominantly follow a discriminative paradigm, which relies heavily on explicit interactions between raw ID embeddings. However, this paradigm inherently renders them susceptible to two critical issues: embedding dimensional collapse and information redundancy, stemming from the over-reliance on feature interactions \emph{over raw ID embeddings}. To address these limitations, we propose a novel \emph{Supervised Feature Generation (SFG)} framework, \emph{shifting the paradigm from discriminative ``feature interaction" to generative ``feature generation"}. Specifically, SFG comprises two key components: an \emph{Encoder} that constructs hidden embeddings for each feature, and a \emph{Decoder} tasked with regenerating the feature embeddings of all features from these hidden representations. Unlike existing generative approaches that adopt self-supervised losses, we introduce a supervised loss to utilize the supervised signal, \ie, click or not, in the CTR prediction task. This framework exhibits strong generalizability: it can be seamlessly integrated with most existing CTR models, reformulating them under the generative paradigm. Extensive experiments demonstrate that SFG consistently mitigates embedding collapse and reduces information redundancy, while yielding substantial performance gains across various datasets and base models. The code is available at https://github.com/USTC-StarTeam/GE4Rec.