FedSGM: A Unified Framework for Constraint Aware, Bidirectionally Compressed, Multi-Step Federated Optimization

TL;DR

FedSGM is a comprehensive federated learning framework that effectively manages constraints, communication efficiency, local updates, and partial participation, backed by theoretical guarantees and experimental validation.

Contribution

FedSGM uniquely unifies constraint handling, compression, multi-step local updates, and partial participation in federated learning with proven convergence guarantees.

Findings

Achieves $oldsymbol{ ext{O}}(1/\sqrt{T})$ convergence rate.

Effectively corrects bias from compression with error feedback.

Validated on Neyman-Pearson classification and CMDP tasks.

Abstract

We introduce FedSGM, a unified framework for federated constrained optimization that addresses four major challenges in federated learning (FL): functional constraints, communication bottlenecks, local updates, and partial client participation. Building on the switching gradient method, FedSGM provides projection-free, primal-only updates, avoiding expensive dual-variable tuning or inner solvers. To handle communication limits, FedSGM incorporates bi-directional error feedback, correcting the bias introduced by compression while explicitly understanding the interaction between compression noise and multi-step local updates. We derive convergence guarantees showing that the averaged iterate achieves the canonical $\boldsymbol{\mathcal{O}}(1/\sqrt{T})$ rate, with additional high-probability bounds that decouple optimization progress from sampling noise due to partial participation. Additionally, we introduce a soft switching version of FedSGM to stabilize updates near the feasibility boundary. To our knowledge, FedSGM is the first framework to unify functional constraints, compression, multiple local updates, and partial client participation, establishing a theoretically grounded foundation for constrained federated learning. Finally, we validate the theoretical guarantees of FedSGM via experimentation on Neyman-Pearson classification and constrained Markov decision process (CMDP) tasks.