Asymptotically Unbiased Estimation for Delayed Feedback Modeling via Label Correction

TL;DR

This paper introduces DEFUSE, a novel method for unbiased delayed feedback modeling in online advertising that corrects importance weights and better utilizes observed positive samples, improving CVR prediction accuracy.

Contribution

The paper proposes DEFUSE, a two-step optimization framework that corrects importance weights and jointly models unbiased positives and biased delays, advancing delayed feedback modeling.

Findings

DEFUSE outperforms existing methods on public and industrial datasets.

The approach effectively corrects bias in delayed feedback modeling.

Experimental results demonstrate significant accuracy improvements.

Abstract

Alleviating the delayed feedback problem is of crucial importance for the conversion rate(CVR) prediction in online advertising. Previous delayed feedback modeling methods using an observation window to balance the trade-off between waiting for accurate labels and consuming fresh feedback. Moreover, to estimate CVR upon the freshly observed but biased distribution with fake negatives, the importance sampling is widely used to reduce the distribution bias. While effective, we argue that previous approaches falsely treat fake negative samples as real negative during the importance weighting and have not fully utilized the observed positive samples, leading to suboptimal performance. In this work, we propose a new method, DElayed Feedback modeling with UnbiaSed Estimation, (DEFUSE), which aim to respectively correct the importance weights of the immediate positive, the fake negative, the real negative, and the delay positive samples at finer granularity. Specifically, we propose a two-step optimization approach that first infers the probability of fake negatives among observed negatives before applying importance sampling. To fully exploit the ground-truth immediate positives from the observed distribution, we further develop a bi-distribution modeling framework to jointly model the unbiased immediate positives and the biased delay conversions. Experimental results on both public and our industrial datasets validate the superiority of DEFUSE. Codes are available at https://github.com/ychen216/DEFUSE.git.