Personalized Federated Learning: A Unified Framework and Universal Optimization Techniques

TL;DR

This paper introduces a unified optimization framework for personalized federated learning, providing general algorithms that improve communication and computation efficiency across various models.

Contribution

It develops universal optimization techniques applicable to many personalized FL objectives, simplifying the design of task-specific optimizers.

Findings

Achieves optimal communication and local computation guarantees.

Provides a comprehensive theory covering many personalized FL models.

Demonstrates practicality of the proposed optimization methods.

Abstract

We investigate the optimization aspects of personalized Federated Learning (FL). We propose general optimizers that can be applied to numerous existing personalized FL objectives, specifically a tailored variant of Local SGD and variants of accelerated coordinate descent/accelerated SVRCD. By examining a general personalized objective capable of recovering many existing personalized FL objectives as special cases, we develop a comprehensive optimization theory applicable to a wide range of strongly convex personalized FL models in the literature. We showcase the practicality and/or optimality of our methods in terms of communication and local computation. Remarkably, our general optimization solvers and theory can recover the best-known communication and computation guarantees for addressing specific personalized FL objectives. Consequently, our proposed methods can serve as universal optimizers, rendering the design of task-specific optimizers unnecessary in many instances.