FedAds: A Benchmark for Privacy-Preserving CVR Estimation with Vertical Federated Learning

TL;DR

FedAds introduces a comprehensive benchmark dataset and evaluation framework for privacy-preserving vertical federated learning in click-through rate prediction, addressing the lack of standardized datasets and systematic assessments.

Contribution

It provides the first large-scale real-world dataset and systematic evaluation methods for vFL-based CVR estimation, including effectiveness and privacy considerations.

Findings

FedAds enables fair comparison of vFL algorithms.

Incorporating unaligned data improves CVR estimation effectiveness.

Perturbation techniques effectively protect user privacy.

Abstract

Conversion rate (CVR) estimation aims to predict the probability of conversion event after a user has clicked an ad. Typically, online publisher has user browsing interests and click feedbacks, while demand-side advertising platform collects users' post-click behaviors such as dwell time and conversion decisions. To estimate CVR accurately and protect data privacy better, vertical federated learning (vFL) is a natural solution to combine two sides' advantages for training models, without exchanging raw data. Both CVR estimation and applied vFL algorithms have attracted increasing research attentions. However, standardized and systematical evaluations are missing: due to the lack of standardized datasets, existing studies adopt public datasets to simulate a vFL setting via hand-crafted feature partition, which brings challenges to fair comparison. We introduce FedAds, the first benchmark for CVR estimation with vFL, to facilitate standardized and systematical evaluations for vFL algorithms. It contains a large-scale real world dataset collected from Alibaba's advertising platform, as well as systematical evaluations for both effectiveness and privacy aspects of various vFL algorithms. Besides, we also explore to incorporate unaligned data in vFL to improve effectiveness, and develop perturbation operations to protect privacy well. We hope that future research work in vFL and CVR estimation benefits from the FedAds benchmark.