A Comparative Evaluation of FedAvg and Per-FedAvg Algorithms for Dirichlet Distributed Heterogeneous Data

TL;DR

This paper compares FedAvg and Per-FedAvg algorithms in federated learning, analyzing their performance on non-IID data modeled by Dirichlet distribution, highlighting Per-FedAvg's robustness under high heterogeneity.

Contribution

It provides a comparative evaluation of FedAvg and Per-FedAvg on Dirichlet-distributed non-IID data, revealing the impact of data heterogeneity on their performance.

Findings

Per-FedAvg outperforms FedAvg in high heterogeneity scenarios.

Data heterogeneity significantly affects federated learning strategies.

Per-FedAvg demonstrates greater robustness to non-IID data.

Abstract

In this paper, we investigate Federated Learning (FL), a paradigm of machine learning that allows for decentralized model training on devices without sharing raw data, there by preserving data privacy. In particular, we compare two strategies within this paradigm: Federated Averaging (FedAvg) and Personalized Federated Averaging (Per-FedAvg), focusing on their performance with Non-Identically and Independently Distributed (Non-IID) data. Our analysis shows that the level of data heterogeneity, modeled using a Dirichlet distribution, significantly affects the performance of both strategies, with Per-FedAvg showing superior robustness in conditions of high heterogeneity. Our results provide insights into the development of more effective and efficient machine learning strategies in a decentralized setting.