Sample-based Federated Learning via Mini-batch SSCA

TL;DR

This paper introduces novel privacy-preserving federated optimization algorithms based on stochastic successive convex approximation (SSCA), capable of handling unconstrained and constrained problems, with proven convergence and practical advantages demonstrated through experiments.

Contribution

First application of SSCA to federated optimization, including nonconvex constrained problems, with customized algorithms and closed-form solutions for efficiency.

Findings

Algorithms converge to KKT points.

Demonstrated faster convergence and lower communication costs.

Effective in practical federated learning scenarios.

Abstract

In this paper, we investigate unconstrained and constrained sample-based federated optimization, respectively. For each problem, we propose a privacy preserving algorithm using stochastic successive convex approximation (SSCA) techniques, and show that it can converge to a Karush-Kuhn-Tucker (KKT) point. To the best of our knowledge, SSCA has not been used for solving federated optimization, and federated optimization with nonconvex constraints has not been investigated. Next, we customize the two proposed SSCA-based algorithms to two application examples, and provide closed-form solutions for the respective approximate convex problems at each iteration of SSCA. Finally, numerical experiments demonstrate inherent advantages of the proposed algorithms in terms of convergence speed, communication cost and model specification.