Federated Residual Learning

TL;DR

This paper introduces a federated learning framework where clients train personalized models alongside a shared server model, reducing complexity and improving robustness to data heterogeneity, with empirical evidence of superior performance.

Contribution

It proposes a novel federated residual learning framework that minimizes shared model complexity while maintaining performance and robustness to non-i.i.d. data.

Findings

Substantial performance improvements over baseline methods.

Robustness to data heterogeneity and slow convergence issues.

Effective personalization within federated learning.

Abstract

We study a new form of federated learning where the clients train personalized local models and make predictions jointly with the server-side shared model. Using this new federated learning framework, the complexity of the central shared model can be minimized while still gaining all the performance benefits that joint training provides. Our framework is robust to data heterogeneity, addressing the slow convergence problem traditional federated learning methods face when the data is non-i.i.d. across clients. We test the theory empirically and find substantial performance gains over baselines.