Towards Federated Learning on Time-Evolving Heterogeneous Data

TL;DR

This paper introduces Continual Federated Learning (CFL), a framework designed to handle the challenges of time-evolving heterogeneity in federated learning, demonstrating faster convergence and superior performance over existing methods.

Contribution

The paper proposes CFL, a novel framework that captures time-evolving heterogeneity in federated learning and provides theoretical and empirical evidence of its advantages.

Findings

CFL converges faster than FedAvg in time-evolving scenarios.

CFL significantly outperforms state-of-the-art FL baselines.

Theoretical analysis confirms improved convergence rates with CFL.

Abstract

Federated Learning (FL) is a learning paradigm that protects privacy by keeping client data on edge devices. However, optimizing FL in practice can be difficult due to the diversity and heterogeneity of the learning system. Despite recent research efforts to improve the optimization of heterogeneous data, the impact of time-evolving heterogeneous data in real-world scenarios, such as changing client data or intermittent clients joining or leaving during training, has not been studied well. In this work, we propose Continual Federated Learning (CFL), a flexible framework for capturing the time-evolving heterogeneity of FL. CFL can handle complex and realistic scenarios, which are difficult to evaluate in previous FL formulations, by extracting information from past local data sets and approximating local objective functions. We theoretically demonstrate that CFL methods have a faster convergence rate than FedAvg in time-evolving scenarios, with the benefit depending on approximation quality. Through experiments, we show that our numerical findings match the convergence analysis and that CFL methods significantly outperform other state-of-the-art FL baselines.