Communication-Efficient Online Federated Learning Framework for Nonlinear Regression

TL;DR

This paper introduces PSO-Fed, a communication-efficient online federated learning framework for kernel regression that allows clients to update models continuously with streaming data while reducing communication costs.

Contribution

The paper proposes a novel partial-sharing-based online federated learning framework (PSO-Fed) that reduces communication overhead in streaming data scenarios for kernel regression.

Findings

PSO-Fed achieves comparable accuracy to existing methods.

It significantly reduces communication overhead.

Theoretical convergence of PSO-Fed is established.

Abstract

Federated learning (FL) literature typically assumes that each client has a fixed amount of data, which is unrealistic in many practical applications. Some recent works introduced a framework for online FL (Online-Fed) wherein clients perform model learning on streaming data and communicate the model to the server; however, they do not address the associated communication overhead. As a solution, this paper presents a partial-sharing-based online federated learning framework (PSO-Fed) that enables clients to update their local models using continuous streaming data and share only portions of those updated models with the server. During a global iteration of PSO-Fed, non-participant clients have the privilege to update their local models with new data. Here, we consider a global task of kernel regression, where clients use a random Fourier features-based kernel LMS on their data for local learning. We examine the mean convergence of the PSO-Fed for kernel regression. Experimental results show that PSO-Fed can achieve competitive performance with a significantly lower communication overhead than Online-Fed.