On Performance Guarantees for Federated Learning with Personalized Constraints

TL;DR

This paper introduces PC-FedAvg, a personalized federated learning method that handles agent-specific constraints, providing theoretical guarantees and demonstrating effectiveness on image classification datasets.

Contribution

It proposes a novel personalized constrained federated optimization algorithm with convergence guarantees and a cross-estimate mechanism for personalization without sharing constraints.

Findings

Achieves communication complexity of O(ε^{-2}) for suboptimality.

Achieves communication complexity of O(ε^{-1}) for infeasibility.

Experimental validation on MNIST and CIFAR-10 datasets supports theoretical results.

Abstract

Federated learning (FL) has emerged as a communication-efficient algorithmic framework for distributed learning across multiple agents. While standard FL formulations capture unconstrained or globally constrained problems, many practical settings involve heterogeneous resource or model constraints, leading to optimization problems with agent-specific feasible sets. Here, we study a personalized constrained federated optimization problem in which each agent is associated with a convex local objective and a private constraint set. We propose PC-FedAvg, a method in which each agent maintains cross-estimates of the other agents' variables through a multi-block local decision vector. Each agent updates all blocks locally, penalizing infeasibility only in its own block. Moreover, the cross-estimate mechanism enables personalization without requiring consensus or sharing constraint information among agents. We establish communication-complexity rates of $\mathcal{O}(\epsilon^{-2})$ for suboptimality and $\mathcal{O}(\epsilon^{-1})$ for agent-wise infeasibility. Preliminary experiments on the MNIST and CIFAR-10 datasets validate our theoretical findings.