Privacy-Preserving Federated Convex Optimization: Balancing Partial-Participation and Efficiency via Noise Cancellation

TL;DR

This paper introduces a noise-cancellation technique for federated convex optimization that ensures differential privacy under partial participation, maintaining efficiency and convergence in distributed learning.

Contribution

It presents a novel noise-cancellation mechanism enabling differential privacy in federated learning with partial participation, extending prior full-participation methods.

Findings

Achieves optimal convergence rates with privacy guarantees in partial-participation settings.

Works effectively for both homogeneous and heterogeneous data distributions.

Maintains computational efficiency comparable to non-private federated optimization.

Abstract

This paper tackles the challenge of achieving Differential Privacy (DP) in Federated Learning (FL) under partial-participation, where only a subset of the machines participate in each time-step. While previous work achieved optimal performance in full-participation settings, these methods struggled to extend to partial-participation scenarios. Our approach fills this gap by introducing a novel noise-cancellation mechanism that preserves privacy without sacrificing convergence rates or computational efficiency. We analyze our method within the Stochastic Convex Optimization (SCO) framework and show that it delivers optimal performance for both homogeneous and heterogeneous data distributions. This work expands the applicability of DP in FL, offering an efficient and practical solution for privacy-preserving learning in distributed systems with partial participation.