NTK-DFL: Enhancing Decentralized Federated Learning in Heterogeneous Settings via Neural Tangent Kernel

TL;DR

This paper introduces NTK-DFL, a novel decentralized federated learning method leveraging neural tangent kernels to improve accuracy and convergence in heterogeneous environments, reducing communication rounds needed.

Contribution

It proposes a new NTK-based approach for decentralized federated learning that enhances performance and convergence speed in heterogeneous data settings.

Findings

Achieves higher accuracy than baselines in heterogeneous environments.

Reaches target performance 4.6 times faster in communication rounds.

Validated across multiple datasets and network topologies.

Abstract

Decentralized federated learning (DFL) is a collaborative machine learning framework for training a model across participants without a central server or raw data exchange. DFL faces challenges due to statistical heterogeneity, as participants often possess data of different distributions reflecting local environments and user behaviors. Recent work has shown that the neural tangent kernel (NTK) approach, when applied to federated learning in a centralized framework, can lead to improved performance. We propose an approach leveraging the NTK to train client models in the decentralized setting, while introducing a synergy between NTK-based evolution and model averaging. This synergy exploits inter-client model deviation and improves both accuracy and convergence in heterogeneous settings. Empirical results demonstrate that our approach consistently achieves higher accuracy than baselines in highly heterogeneous settings, where other approaches often underperform. Additionally, it reaches target performance in 4.6 times fewer communication rounds. We validate our approach across multiple datasets, network topologies, and heterogeneity settings to ensure robustness and generalization.