Differentially Private Federated Clustering with Random Rebalancing

TL;DR

This paper introduces RR-Cluster, a simple rebalancing technique for federated clustering that reduces privacy noise and improves utility while maintaining privacy guarantees, validated through theoretical analysis and experiments.

Contribution

The paper proposes RR-Cluster, a lightweight rebalancing method that enhances privacy-utility tradeoffs in federated clustering by controlling cluster assignment sizes.

Findings

RR-Cluster reduces privacy noise variance.

Improves clustering utility under differential privacy.

Demonstrates effectiveness on synthetic and real datasets.

Abstract

Federated clustering aims to group similar clients into clusters and produce one model for each cluster. Such a personalization approach typically improves model performance compared with training a single model to serve all clients, but can be more vulnerable to privacy leakage. Directly applying client-level differentially private (DP) mechanisms to federated clustering could degrade the utilities significantly. We identify that such deficiencies are mainly due to the difficulties of averaging privacy noise within each cluster (following standard privacy mechanisms), as the number of clients assigned to the same clusters is uncontrolled. To this end, we propose a simple and effective technique, named RR-Cluster, that can be viewed as a light-weight add-on to many federated clustering algorithms. RR-Cluster achieves reduced privacy noise via randomly rebalancing cluster assignments, guaranteeing a minimum number of clients assigned to each cluster. We analyze the tradeoffs between decreased privacy noise variance and potentially increased bias from incorrect assignments and provide convergence bounds for RR-Clsuter. Empirically, we demonstrate the RR-Cluster plugged into strong federated clustering algorithms results in significantly improved privacy/utility tradeoffs across both synthetic and real-world datasets.