Sharper Convergence Guarantees for Federated Learning with Partial Model Personalization

TL;DR

This paper introduces two algorithms, FedAvg-P and Scaffold-P, for partial model personalization in federated learning, providing sharper convergence guarantees and improved theoretical rates under relaxed assumptions.

Contribution

It proposes novel algorithms for partial model personalization in federated learning with rigorous convergence analysis and improved theoretical guarantees.

Findings

Established convergence rates surpass existing results.

Analysis applies to fully shared or personalized federated learning.

Numerical experiments confirm theoretical improvements.

Abstract

Partial model personalization, which encompasses both shared and personal variables in its formulation, is a critical optimization problem in federated learning. It balances individual client needs with collective knowledge utilization, and serves as a general formulation covering various key scenarios, ranging from fully shared to fully personalized federated learning. This paper introduces two effective algorithms, FedAvg-P and Scaffold-P, to solve this problem and provides sharp convergence analyses, quantifying the influence of gradient variance, local steps, and partial client sampling on their performance. Our established rates surpass existing results and, meanwhile, are based on more relaxed assumptions. Additionally, our analyses are also applicable to fully shared or fully personalized federated learning, matching or even outperforming their best known convergence rates. Numerical experiments corroborate our theoretical findings.