Noise-Robust and Resource-Efficient ADMM-based Federated Learning

TL;DR

This paper introduces a noise-robust and resource-efficient federated learning algorithm based on ADMM, which improves model accuracy under communication noise and reduces communication overhead through innovative modifications and theoretical guarantees.

Contribution

The paper proposes a novel ADMM-based federated learning algorithm that enhances robustness to communication noise and reduces communication load by eliminating dual variables and allowing continuous local updates.

Findings

The algorithm converges in mean and mean-square senses under noisy, random client-server communication.

Numerical results demonstrate improved robustness and efficiency compared to existing methods.

Theoretical analysis confirms convergence despite communication noise and random client selection.

Abstract

Federated learning (FL) leverages client-server communications to train global models on decentralized data. However, communication noise or errors can impair model accuracy. To address this problem, we propose a novel FL algorithm that enhances robustness against communication noise while also reducing communication load. We derive the proposed algorithm through solving the weighted least-squares (WLS) regression problem as an illustrative example. We first frame WLS regression as a distributed convex optimization problem over a federated network employing random scheduling for improved communication efficiency. We then apply the alternating direction method of multipliers (ADMM) to iteratively solve this problem. To counteract the detrimental effects of cumulative communication noise, we introduce a key modification by eliminating the dual variable and implementing a new local model update at each participating client. This subtle yet effective change results in using a single noisy global model update at each client instead of two, improving robustness against additive communication noise. Furthermore, we incorporate another modification enabling clients to continue local updates even when not selected by the server, leading to substantial performance improvements. Our theoretical analysis confirms the convergence of our algorithm in both mean and the mean-square senses, even when the server communicates with a random subset of clients over noisy links at each iteration. Numerical results validate the effectiveness of our proposed algorithm and corroborate our theoretical findings.