Generative Click-through Rate Prediction with Applications to Search Advertising

TL;DR

This paper introduces a novel generative model-based approach for CTR prediction in search advertising, combining generative pre-training and discriminative fine-tuning, leading to improved accuracy validated by extensive experiments and online A/B testing.

Contribution

The paper proposes a new two-stage training method that integrates generative models into CTR prediction, enhancing predictive performance over traditional discriminative models.

Findings

Significant improvement in CTR prediction accuracy demonstrated.

Effective deployment on a large-scale e-commerce platform.

Positive results from online A/B testing confirm utility.

Abstract

Click-Through Rate (CTR) prediction models are integral to a myriad of industrial settings, such as personalized search advertising. Current methods typically involve feature extraction from users' historical behavior sequences combined with product information, feeding into a discriminative model that is trained on user feedback to estimate CTR. With the success of models such as GPT, the potential for generative models to enrich expressive power beyond discriminative models has become apparent. In light of this, we introduce a novel model that leverages generative models to enhance the precision of CTR predictions in discriminative models. To reconcile the disparate data aggregation needs of both model types, we design a two-stage training process: 1) Generative pre-training for next-item prediction with the given item category in user behavior sequences; 2) Fine-tuning the well-trained generative model within a discriminative CTR prediction framework. Our method's efficacy is substantiated through extensive experiments on a new dataset, and its significant utility is further corroborated by online A/B testing results. Currently, the model is deployed on one of the world's largest e-commerce platforms, and we intend to release the associated code and dataset in the future.