Federated Frank-Wolfe Algorithm

TL;DR

The paper introduces FedFW, a federated learning algorithm using Frank-Wolfe that efficiently handles constrained problems with privacy, low cost, and sparse communication, suitable for both convex and non-convex objectives.

Contribution

It proposes FedFW, a novel federated Frank-Wolfe algorithm that reduces projection costs and supports both deterministic and stochastic settings for constrained machine learning.

Findings

FedFW achieves $ ext{ extsterling}$-suboptimal solutions within $O( ext{ extsterling}^{-2})$ iterations for convex objectives.

FedFW converges within $O( ext{ extsterling}^{-3})$ iterations for non-convex objectives.

Empirical results demonstrate FedFW's effectiveness on various machine learning tasks.

Abstract

Federated learning (FL) has gained a lot of attention in recent years for building privacy-preserving collaborative learning systems. However, FL algorithms for constrained machine learning problems are still limited, particularly when the projection step is costly. To this end, we propose a Federated Frank-Wolfe Algorithm (FedFW). FedFW features data privacy, low per-iteration cost, and communication of sparse signals. In the deterministic setting, FedFW achieves an $\varepsilon$-suboptimal solution within $O(\varepsilon^{-2})$ iterations for smooth and convex objectives, and $O(\varepsilon^{-3})$ iterations for smooth but non-convex objectives. Furthermore, we present a stochastic variant of FedFW and show that it finds a solution within $O(\varepsilon^{-3})$ iterations in the convex setting. We demonstrate the empirical performance of FedFW on several machine learning tasks.