UKD: Debiasing Conversion Rate Estimation via Uncertainty-regularized Knowledge Distillation

TL;DR

This paper introduces UKD, a novel framework that uses uncertainty-regularized knowledge distillation to improve conversion rate estimation by leveraging unclicked ads, effectively reducing sample selection bias.

Contribution

The paper proposes a new UKD framework that distills knowledge from unclicked ads with uncertainty modeling to debias CVR estimation, a novel approach in online advertising.

Findings

UKD outperforms previous debiasing methods on large datasets.

UKD achieves significant online performance improvements.

Uncertainty modeling effectively handles noise in pseudo-labels.

Abstract

In online advertising, conventional post-click conversion rate (CVR) estimation models are trained using clicked samples. However, during online serving the models need to estimate for all impression ads, leading to the sample selection bias (SSB) issue. Intuitively, providing reliable supervision signals for unclicked ads is a feasible way to alleviate the SSB issue. This paper proposes an uncertainty-regularized knowledge distillation (UKD) framework to debias CVR estimation via distilling knowledge from unclicked ads. A teacher model learns click-adaptive representations and produces pseudo-conversion labels on unclicked ads as supervision signals. Then a student model is trained on both clicked and unclicked ads with knowledge distillation, performing uncertainty modeling to alleviate the inherent noise in pseudo-labels. Experiments on billion-scale datasets show that UKD outperforms previous debiasing methods. Online results verify that UKD achieves significant improvements.