Implicit Model Specialization through DAG-based Decentralized Federated Learning

TL;DR

This paper introduces a DAG-based decentralized federated learning approach that enables clients to develop specialized models tailored to their local data, improving performance on non-IID data and enhancing robustness.

Contribution

It presents a novel DAG-based framework that unifies decentralization, personalization, and poisoning robustness in federated learning, a first in the field.

Findings

Specialized models emerge from DAG communication.

Stable accuracy and reduced variance across clients.

Improved handling of non-IID data.

Abstract

Federated learning allows a group of distributed clients to train a common machine learning model on private data. The exchange of model updates is managed either by a central entity or in a decentralized way, e.g. by a blockchain. However, the strong generalization across all clients makes these approaches unsuited for non-independent and identically distributed (non-IID) data. We propose a unified approach to decentralization and personalization in federated learning that is based on a directed acyclic graph (DAG) of model updates. Instead of training a single global model, clients specialize on their local data while using the model updates from other clients dependent on the similarity of their respective data. This specialization implicitly emerges from the DAG-based communication and selection of model updates. Thus, we enable the evolution of specialized models, which focus on a subset of the data and therefore cover non-IID data better than federated learning in a centralized or blockchain-based setup. To the best of our knowledge, the proposed solution is the first to unite personalization and poisoning robustness in fully decentralized federated learning. Our evaluation shows that the specialization of models emerges directly from the DAG-based communication of model updates on three different datasets. Furthermore, we show stable model accuracy and less variance across clients when compared to federated averaging.