Efficient Node Selection in Private Personalized Decentralized Learning

TL;DR

This paper introduces PPDL, a privacy-preserving decentralized learning method that uses secure aggregation and multi-armed bandit optimization to select collaborators effectively, improving model performance while safeguarding sensitive data.

Contribution

The paper presents PPDL, a novel private decentralized learning framework combining secure aggregation with correlated bandit optimization for effective node selection.

Findings

PPDL outperforms non-private methods on standard benchmarks.

PPDL effectively identifies suitable collaborators using only aggregated models.

PPDL maintains privacy while improving model accuracy under data shift scenarios.

Abstract

Personalized decentralized learning is a promising paradigm for distributed learning, enabling each node to train a local model on its own data and collaborate with other nodes to improve without sharing any data. However, this approach poses significant privacy risks, as nodes may inadvertently disclose sensitive information about their data or preferences through their collaboration choices. In this paper, we propose Private Personalized Decentralized Learning (PPDL), a novel approach that combines secure aggregation and correlated adversarial multi-armed bandit optimization to protect node privacy while facilitating efficient node selection. By leveraging dependencies between different arms, represented by potential collaborators, we demonstrate that PPDL can effectively identify suitable collaborators solely based on aggregated models. Additionally, we show that PPDL surpasses previous non-private methods in model performance on standard benchmarks under label and covariate shift scenarios.