Private and Federated Stochastic Convex Optimization: Efficient Strategies for Centralized Systems

TL;DR

This paper develops efficient differentially private stochastic convex optimization methods for federated learning in centralized systems, ensuring privacy without sacrificing convergence speed or computational efficiency across various data distributions.

Contribution

It introduces a novel approach that achieves optimal convergence and differential privacy in federated learning with linear complexity, applicable to both trusted and untrusted server scenarios.

Findings

Achieves differential privacy with optimal convergence rates.

Maintains linear computational complexity similar to non-private methods.

Effective in both homogeneous and heterogeneous data settings.

Abstract

This paper addresses the challenge of preserving privacy in Federated Learning (FL) within centralized systems, focusing on both trusted and untrusted server scenarios. We analyze this setting within the Stochastic Convex Optimization (SCO) framework, and devise methods that ensure Differential Privacy (DP) while maintaining optimal convergence rates for homogeneous and heterogeneous data distributions. Our approach, based on a recent stochastic optimization technique, offers linear computational complexity, comparable to non-private FL methods, and reduced gradient obfuscation. This work enhances the practicality of DP in FL, balancing privacy, efficiency, and robustness in a variety of server trust environment.