On Bridging Generic and Personalized Federated Learning for Image Classification

TL;DR

This paper introduces Fed-RoD, a federated learning framework that effectively balances generic and personalized image classification by decoupling model tasks, achieving state-of-the-art results for both objectives.

Contribution

It proposes a novel dual-task federated learning framework that explicitly separates generic and personalized predictors, enabling simultaneous optimization of both.

Findings

Achieves state-of-the-art generic performance.

Attains superior personalized accuracy.

Effectively balances both objectives in federated learning.

Abstract

Federated learning is promising for its capability to collaboratively train models with multiple clients without accessing their data, but vulnerable when clients' data distributions diverge from each other. This divergence further leads to a dilemma: "Should we prioritize the learned model's generic performance (for future use at the server) or its personalized performance (for each client)?" These two, seemingly competing goals have divided the community to focus on one or the other, yet in this paper we show that it is possible to approach both at the same time. Concretely, we propose a novel federated learning framework that explicitly decouples a model's dual duties with two prediction tasks. On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them. On the other hand, we formulate the personalized predictor as a lightweight adaptive module that is learned to minimize each client's empirical risk on top of the generic predictor. With this two-loss, two-predictor framework which we name Federated Robust Decoupling (Fed-RoD), the learned model can simultaneously achieve state-of-the-art generic and personalized performance, essentially bridging the two tasks.