FedNMUT -- Federated Noisy Model Update Tracking Convergence Analysis

TL;DR

FedNMUT is a decentralized federated learning algorithm designed to maintain convergence and robustness in noisy communication environments by incorporating noise modeling, gradient tracking, and theoretical convergence guarantees.

Contribution

It introduces FedNMUT, a novel decentralized federated learning method that effectively handles noisy channels and data heterogeneity with proven convergence properties.

Findings

FedNMUT achieves an $oldsymbol{ extit{ ext{O}}}(rac{1}{ ext{sqrt}(T)})$ convergence rate.

It outperforms existing methods in noisy communication scenarios.

Theoretical analysis confirms convergence for non-convex objectives.

Abstract

A novel Decentralized Noisy Model Update Tracking Federated Learning algorithm (FedNMUT) is proposed that is tailored to function efficiently in the presence of noisy communication channels that reflect imperfect information exchange. This algorithm uses gradient tracking to minimize the impact of data heterogeneity while minimizing communication overhead. The proposed algorithm incorporates noise into its parameters to mimic the conditions of noisy communication channels, thereby enabling consensus among clients through a communication graph topology in such challenging environments. FedNMUT prioritizes parameter sharing and noise incorporation to increase the resilience of decentralized learning systems against noisy communications. Theoretical results for the smooth non-convex objective function are provided by us, and it is shown that the $\epsilon-$stationary solution is achieved by our algorithm at the rate of $\mathcal{O}\left(\frac{1}{\sqrt{T}}\right)$, where $T$ is the total number of communication rounds. Additionally, via empirical validation, we demonstrated that the performance of FedNMUT is superior to the existing state-of-the-art methods and conventional parameter-mixing approaches in dealing with imperfect information sharing. This proves the capability of the proposed algorithm to counteract the negative effects of communication noise in a decentralized learning framework.