Improved Convergence Analysis and SNR Control Strategies for Federated Learning in the Presence of Noise

TL;DR

This paper introduces an improved convergence analysis for federated learning under noisy communication channels, revealing the asymmetric impact of uplink and downlink noise, and proposes SNR control strategies to maintain optimal convergence with reduced power consumption.

Contribution

It provides the first analysis showing the asymmetry of uplink and downlink noise effects in FL and proposes SNR control strategies that achieve near-noiseless convergence rates with less power.

Findings

Downlink noise has a more severe impact on FL convergence.

Scaling down uplink noise by √k and downlink noise by k maintains O(1/√K) convergence.

The analysis does not assume bounded client dissimilarity and applies to smooth non-convex functions.

Abstract

We propose an improved convergence analysis technique that characterizes the distributed learning paradigm of federated learning (FL) with imperfect/noisy uplink and downlink communications. Such imperfect communication scenarios arise in the practical deployment of FL in emerging communication systems and protocols. The analysis developed in this paper demonstrates, for the first time, that there is an asymmetry in the detrimental effects of uplink and downlink communications in FL. In particular, the adverse effect of the downlink noise is more severe on the convergence of FL algorithms. Using this insight, we propose improved Signal-to-Noise (SNR) control strategies that, discarding the negligible higher-order terms, lead to a similar convergence rate for FL as in the case of a perfect, noise-free communication channel while incurring significantly less power resources compared to existing solutions. In particular, we establish that to maintain the $O(\frac{1}{\sqrt{K}})$ rate of convergence like in the case of noise-free FL, we need to scale down the uplink and downlink noise by $\Omega({\sqrt{k}})$ and $\Omega({k})$ respectively, where $k$ denotes the communication round, $k=1,\dots, K$. Our theoretical result is further characterized by two major benefits: firstly, it does not assume the somewhat unrealistic assumption of bounded client dissimilarity, and secondly, it only requires smooth non-convex loss functions, a function class better suited for modern machine learning and deep learning models. We also perform extensive empirical analysis to verify the validity of our theoretical findings.