On the Fast Adaptation of Delayed Clients in Decentralized Federated Learning: A Centroid-Aligned Distillation Approach

TL;DR

This paper introduces DFedCAD, a framework that enables rapid adaptation of delayed clients in decentralized federated learning through centroid-aligned distillation, significantly reducing communication costs and improving accuracy.

Contribution

The paper proposes a novel centroid-aligned distillation method with weighted cluster pruning for efficient knowledge transfer in asynchronous decentralized federated learning.

Findings

Achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, Tiny-ImageNet.

Reduces communication overhead by over 86%.

Effectively adapts delayed clients in dynamic environments.

Abstract

Decentralized Federated Learning (DFL) struggles with the slow adaptation of late-joining delayed clients and high communication costs in asynchronous environments. These limitations significantly hinder overall performance. To address this, we propose DFedCAD, a novel framework for rapid adaptation via Centroid-Aligned Distillation. DFedCAD first employs Weighted Cluster Pruning (WCP) to compress models into representative centroids, drastically reducing communication overhead. It then enables delayed clients to intelligently weigh and align with peer knowledge using a novel structural distance metric and a differentiable k-means distillation module, facilitating efficient end-to-end knowledge transfer. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that DFedCAD consistently achieves state-of-the-art performance, attaining the highest accuracy across all evaluated settings while reducing communication overhead by over 86%. Our framework provides a scalable and practical solution for efficient decentralized learning in dynamic, real-world scenarios.