Online-Score-Aided Federated Learning: Taming the Resource Constraints in Wireless Networks

TL;DR

This paper introduces OSAFL, a federated learning algorithm tailored for wireless networks with resource constraints and online data, improving convergence without extra communication overheads.

Contribution

The paper proposes a novel online-score-aided federated learning algorithm that accounts for resource limitations and online data, enhancing convergence in wireless network scenarios.

Findings

OSAFL outperforms existing FL methods in simulations.

Theoretical analysis shows conditions for sublinear convergence.

Effective handling of resource constraints and data shifts.

Abstract

While federated learning (FL) is a widely popular distributed machine learning (ML) strategy that protects data privacy, time-varying wireless network parameters and heterogeneous configurations of the wireless devices pose significant challenges. Although the limited radio and computational resources of the network and the clients, respectively, are widely acknowledged, two critical yet often ignored aspects are (a) wireless devices can only dedicate a small chunk of their limited storage for the FL task and (b) new training samples may arrive in an online manner in many practical wireless applications. Therefore, we propose a new FL algorithm called online-score-aided federated learning (OSAFL), specifically designed to learn tasks relevant to wireless applications under these practical considerations. Since clients' local training steps differ under resource constraints, which may lead to client drift under statistically heterogeneous data distributions, we leverage normalized gradient similarities and exploit weighting clients' updates based on optimized scores that facilitate the convergence rate of the proposed OSAFL algorithm without incurring any communication overheads to the clients or requiring any statistical data information from them. We theoretically show how the new factors, i.e., online score and local data distribution shifts, affect the convergence bound and derive the necessary conditions for a sublinear convergence rate. Our extensive simulation results on two different tasks with multiple popular ML models validate the effectiveness of the proposed OSAFL algorithm compared to modified state-of-the-art FL baselines.