Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

TL;DR

This paper introduces a Customized Federated Learning system that addresses heterogeneity in edge devices, significantly improving accuracy, efficiency, and fairness in federated learning scenarios.

Contribution

It proposes a multidimensional personalization approach with a new aggregation algorithm and model-search helper, advancing federated learning performance and fairness.

Findings

Up to 7.2% accuracy improvement in non-heterogeneous environments.

Up to 21.8% accuracy improvement in heterogeneous environments.

Enhanced fairness and efficiency in federated learning.

Abstract

Federated learning (FL) is an emerging technique that trains massive and geographically distributed edge data while maintaining privacy. However, FL has inherent challenges in terms of fairness and computational efficiency due to the rising heterogeneity of edges, and thus usually results in sub-optimal performance in recent state-of-the-art (SOTA) solutions. In this paper, we propose a Customized Federated Learning (CFL) system to eliminate FL heterogeneity from multiple dimensions. Specifically, CFL tailors personalized models from the specially designed global model for each client jointly guided by an online trained model-search helper and a novel aggregation algorithm. Extensive experiments demonstrate that CFL has full-stack advantages for both FL training and edge reasoning and significantly improves the SOTA performance w.r.t. model accuracy (up to 7.2% in the non-heterogeneous environment and up to 21.8% in the heterogeneous environment), efficiency, and FL fairness.