VFed-SSD: Towards Practical Vertical Federated Advertising

TL;DR

This paper introduces VFed-SSD, a semi-supervised federated learning framework for advertising that leverages unlabeled data and model decomposition to improve performance while reducing deployment costs.

Contribution

The paper proposes a novel semi-supervised split distillation framework for vertical federated learning in advertising, addressing data scarcity and real-time serving challenges.

Findings

Median AUC improved by 0.86% and 2.6% in local and federated modes

Effective utilization of unlabeled data via self-supervised task MPD

Reduced deployment cost with maintained or improved model performance

Abstract

As an emerging secure learning paradigm in lever-aging cross-agency private data, vertical federatedlearning (VFL) is expected to improve advertising models by enabling the joint learning of complementary user attributes privately owned by the advertiser and the publisher. However, there are two key challenges in applying it to advertising systems: a) the limited scale of labeled overlapping samples, and b) the high cost of real-time cross-agency serving. In this paper, we propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. We identify that: i)there are massive unlabeled overlapped data available in advertising systems, and ii) we can keep a balance between model performance and inference cost by decomposing the federated model. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data and propose the Split Knowledge Distillation (SplitKD) schema to avoid cross-agency serving. Empirical studies on three industrial datasets exhibit the effectiveness of ourmethods, with the median AUC over all datasets improved by 0.86% and 2.6% in the local andthe federated deployment mode respectively. Overall, our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.