FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation

TL;DR

FedPDD introduces a privacy-preserving double distillation framework for cross-silo federated recommendation, effectively leveraging limited overlapping user data while ensuring privacy and reducing communication costs.

Contribution

The paper proposes a novel double distillation approach with offline training and differential privacy for improved federated recommendation with limited user overlap.

Findings

Outperforms state-of-the-art methods on real-world datasets

Reduces communication costs through offline training scheme

Enhances privacy with differential privacy mechanisms

Abstract

Cross-platform recommendation aims to improve recommendation accuracy by gathering heterogeneous features from different platforms. However, such cross-silo collaborations between platforms are restricted by increasingly stringent privacy protection regulations, thus data cannot be aggregated for training. Federated learning (FL) is a practical solution to deal with the data silo problem in recommendation scenarios. Existing cross-silo FL methods transmit model information to collaboratively build a global model by leveraging the data of overlapped users. However, in reality, the number of overlapped users is often very small, thus largely limiting the performance of such approaches. Moreover, transmitting model information during training requires high communication costs and may cause serious privacy leakage. In this paper, we propose a novel privacy-preserving double distillation framework named FedPDD for cross-silo federated recommendation, which efficiently transfers knowledge when overlapped users are limited. Specifically, our double distillation strategy enables local models to learn not only explicit knowledge from the other party but also implicit knowledge from its past predictions. Moreover, to ensure privacy and high efficiency, we employ an offline training scheme to reduce communication needs and privacy leakage risk. In addition, we adopt differential privacy to further protect the transmitted information. The experiments on two real-world recommendation datasets, HetRec-MovieLens and Criteo, demonstrate the effectiveness of FedPDD compared to the state-of-the-art approaches.