Fractional-Order Federated Learning

TL;DR

This paper introduces FOFedAvg, a fractional-order federated learning algorithm that enhances convergence speed and robustness by incorporating memory-aware fractional updates, proven to converge under standard assumptions.

Contribution

It proposes a novel fractional-order federated averaging method that improves communication efficiency and convergence in non-IID data settings, with theoretical convergence guarantees.

Findings

FOFedAvg outperforms baseline algorithms on multiple benchmark datasets.

Fractional-order updates improve communication efficiency and convergence speed.

Theoretical proof of convergence under standard assumptions.

Abstract

Federated learning (FL) allows remote clients to train a global model collaboratively while protecting client privacy. Despite its privacy-preserving benefits, FL has significant drawbacks, including slow convergence, high communication cost, and non-independent-and-identically-distributed (non-IID) data. In this work, we present a novel FedAvg variation called Fractional-Order Federated Averaging (FOFedAvg), which incorporates Fractional-Order Stochastic Gradient Descent (FOSGD) to capture long-range relationships and deeper historical information. By introducing memory-aware fractional-order updates, FOFedAvg improves communication efficiency and accelerates convergence while mitigating instability caused by heterogeneous, non-IID client data. We compare FOFedAvg against a broad set of established federated optimization algorithms on benchmark datasets including MNIST, FEMNIST, CIFAR-10, CIFAR-100, EMNIST, the Cleveland heart disease dataset, Sent140, PneumoniaMNIST, and Edge-IIoTset. Across a range of non-IID partitioning schemes, FOFedAvg is competitive with, and often outperforms, these baselines in terms of test performance and convergence speed. On the theoretical side, we prove that FOFedAvg converges to a stationary point under standard smoothness and bounded-variance assumptions for fractional order $0<\alpha\le 1$. Together, these results show that fractional-order, memory-aware updates can substantially improve the robustness and effectiveness of federated learning, offering a practical path toward distributed training on heterogeneous data.