Mitigating Persistent Client Dropout in Asynchronous Decentralized Federated Learning

TL;DR

This paper addresses persistent client dropout in asynchronous decentralized federated learning, proposing adaptive reconstruction strategies that improve robustness despite limited information and data heterogeneity.

Contribution

It introduces novel adaptive strategies for client reconstruction in asynchronous DFL, enhancing robustness against dropout without requiring precise data recovery.

Findings

Adaptive strategies recover performance loss due to dropout.

Strategies are effective across various datasets and heterogeneity scenarios.

The approach maintains robustness even with limited information.

Abstract

We consider the problem of persistent client dropout in asynchronous Decentralized Federated Learning (DFL). Asynchronicity and decentralization obfuscate information about model updates among federation peers, making recovery from a client dropout difficult. Access to the number of learning epochs, data distributions, and all the information necessary to precisely reconstruct the missing neighbor's loss functions is limited. We show that obvious mitigations do not adequately address the problem and introduce adaptive strategies based on client reconstruction. We show that these strategies can effectively recover some performance loss caused by dropout. Our work focuses on asynchronous DFL with local regularization and differs substantially from that in the existing literature. We evaluate the proposed methods on tabular and image datasets, involve three DFL algorithms, and three data heterogeneity scenarios (iid, non-iid, class-focused non-iid). Our experiments show that the proposed adaptive strategies can be effective in maintaining robustness of federated learning, even if they do not reconstruct the missing client's data precisely. We also discuss the limitations and identify future avenues for tackling the problem of client dropout.