A General Framework for Debiasing in CTR Prediction

TL;DR

This paper introduces a versatile debiasing framework for CTR prediction that overcomes the limitations of existing methods by modeling complex relationships, leading to improved performance across various scenarios in both simulations and online settings.

Contribution

A novel general debiasing framework that does not rely on simplifying assumptions, applicable to diverse CTR prediction scenarios.

Findings

Maintains similar AUC in simple scenarios compared to state-of-the-art methods.

Achieves significant improvements in complex scenarios.

Consistently improves online CTR prediction performance.

Abstract

Most of the existing methods for debaising in click-through rate (CTR) prediction depend on an oversimplified assumption, i.e., the click probability is the product of observation probability and relevance probability. However, since there is a complicated interplay between these two probabilities, these methods cannot be applied to other scenarios, e.g. query auto completion (QAC) and route recommendation. We propose a general debiasing framework without simplifying the relationships between variables, which can handle all scenarios in CTR prediction. Simulation experiments show that: under the simplest scenario, our method maintains a similar AUC with the state-of-the-art methods; in other scenarios, our method achieves considerable improvements compared with existing methods. Meanwhile, in online experiments, the framework also gains significant improvements consistently.