FBFL: A Field-Based Coordination Approach for Data Heterogeneity in Federated Learning

TL;DR

This paper introduces FBFL, a novel field-based federated learning approach that enhances scalability, personalization, and robustness in non-IID data environments through macroprogramming and hierarchical coordination.

Contribution

FBFL is the first to apply macroprogramming and field coordination to federated learning, enabling distributed leader election and self-organizing architectures for improved performance and resilience.

Findings

FBFL performs comparably to FedAvg under IID data.

FBFL outperforms FedAvg, FedProx, and Scaffold in non-IID scenarios.

FBFL's architecture is resilient to server failures.

Abstract

In the last years, Federated learning (FL) has become a popular solution to train machine learning models in domains with high privacy concerns. However, FL scalability and performance face significant challenges in real-world deployments where data across devices are non-independently and identically distributed (non-IID). The heterogeneity in data distribution frequently arises from spatial distribution of devices, leading to degraded model performance in the absence of proper handling. Additionally, FL typical reliance on centralized architectures introduces bottlenecks and single-point-of-failure risks, particularly problematic at scale or in dynamic environments. To close this gap, we propose Field-Based Federated Learning (FBFL), a novel approach leveraging macroprogramming and field coordination to address these limitations through: (i) distributed spatial-based leader election for personalization to mitigate non-IID data challenges; and (ii) construction of a self-organizing, hierarchical architecture using advanced macroprogramming patterns. Moreover, FBFL not only overcomes the aforementioned limitations, but also enables the development of more specialized models tailored to the specific data distribution in each subregion. This paper formalizes FBFL and evaluates it extensively using MNIST, FashionMNIST, and Extended MNIST datasets. We demonstrate that, when operating under IID data conditions, FBFL performs comparably to the widely-used FedAvg algorithm. Furthermore, in challenging non-IID scenarios, FBFL not only outperforms FedAvg but also surpasses other state-of-the-art methods, namely FedProx and Scaffold, which have been specifically designed to address non-IID data distributions. Additionally, we showcase the resilience of FBFL's self-organizing hierarchical architecture against server failures.