Federated Online Learning for Heterogeneous Multisource Streaming Data

TL;DR

This paper introduces a federated online learning method tailored for multi-source streaming data, addressing heterogeneity and privacy concerns while ensuring efficient, consistent, and accurate model estimation and prediction.

Contribution

It proposes a novel federated online learning framework with personalized models and subgroup assumptions, improving performance on heterogeneous streaming data.

Findings

The method achieves consistent model estimation and subgroup recovery.

It demonstrates superior prediction accuracy on real-world datasets.

The approach reduces storage needs by using summary statistics.

Abstract

Federated learning has emerged as an essential paradigm for distributed multi-source data analysis under privacy concerns. Most existing federated learning methods focus on the ``static" datasets. However, in many real-world applications, data arrive continuously over time, forming streaming datasets. This introduces additional challenges for data storage and algorithm design, particularly under high-dimensional settings. In this paper, we propose a federated online learning (FOL) method for distributed multi-source streaming data analysis. To account for heterogeneity, a personalized model is constructed for each data source, and a novel ``subgroup" assumption is employed to capture potential similarities, thereby enhancing model performance. We adopt the penalized renewable estimation method and the efficient proximal gradient descent for model training. The proposed method aligns with both federated and online learning frameworks: raw data are not exchanged among sources, ensuring data privacy, and only summary statistics of previous data batches are required for model updates, significantly reducing storage demands. Theoretically, we establish the consistency properties for model estimation, variable selection, and subgroup structure recovery, demonstrating optimal statistical efficiency. Simulations illustrate the effectiveness of the proposed method. Furthermore, when applied to the financial lending data and the web log data, the proposed method also exhibits advantageous prediction performance. Results of the analysis also provide some practical insights.