Enhancing Federated Learning with Kolmogorov-Arnold Networks: A Comparative Study Across Diverse Aggregation Strategies

TL;DR

This study demonstrates that Kolmogorov-Arnold Networks outperform traditional MLPs in federated learning across diverse datasets, showing improved accuracy, stability, convergence speed, and robustness under client heterogeneity.

Contribution

The paper introduces the application of Kolmogorov-Arnold Networks in federated learning and compares their performance to MLPs across various aggregation strategies and data conditions.

Findings

KANs outperform MLPs in accuracy, stability, and convergence.

KANs are robust under varying client numbers and non-IID data.

Fewer communication rounds are needed for KANs to converge.

Abstract

Multilayer Perceptron (MLP), as a simple yet powerful model, continues to be widely used in classification and regression tasks. However, traditional MLPs often struggle to efficiently capture nonlinear relationships in load data when dealing with complex datasets. Kolmogorov-Arnold Networks (KAN), inspired by the Kolmogorov-Arnold representation theorem, have shown promising capabilities in modeling complex nonlinear relationships. In this study, we explore the performance of KANs within federated learning (FL) frameworks and compare them to traditional Multilayer Perceptrons. Our experiments, conducted across four diverse datasets demonstrate that KANs consistently outperform MLPs in terms of accuracy, stability, and convergence efficiency. KANs exhibit remarkable robustness under varying client numbers and non-IID data distributions, maintaining superior performance even as client heterogeneity increases. Notably, KANs require fewer communication rounds to converge compared to MLPs, highlighting their efficiency in FL scenarios. Additionally, we evaluate multiple parameter aggregation strategies, with trimmed mean and FedProx emerging as the most effective for optimizing KAN performance. These findings establish KANs as a robust and scalable alternative to MLPs for federated learning tasks, paving the way for their application in decentralized and privacy-preserving environments.