FlexFed: Mitigating Catastrophic Forgetting in Heterogeneous Federated Learning in Pervasive Computing Environments

TL;DR

FlexFed is a novel federated learning method designed for resource-constrained pervasive computing environments, effectively mitigating catastrophic forgetting by adaptive data retention and training strategies, leading to improved stability and efficiency.

Contribution

The paper introduces FlexFed, a new FL approach that addresses catastrophic forgetting in HAR settings without privacy-compromising replay mechanisms, using dynamic training adjustments and a new CF metric.

Findings

FlexFed reduces catastrophic forgetting more effectively than existing methods.

It improves federated learning efficiency by 10-15%.

Achieves faster, more stable convergence especially for infrequent data.

Abstract

Federated Learning (FL) enables collaborative model training while preserving privacy by allowing clients to share model updates instead of raw data. Pervasive computing environments (e.g., for Human Activity Recognition, HAR), which we focus on in this paper, are characterized by resource-constrained end devices, streaming sensor data and intermittent client participation. Variations in user behavior, common in HAR environments, often result in non-stationary data distributions. As such, existing FL approaches face challenges in HAR settings due to differing assumptions. The combined effects of HAR characteristics, namely heterogeneous data and intermittent participation, can lead to a severe issue called catastrophic forgetting (CF). Unlike Continuous Learning (CL), which addresses CF using memory and replay mechanisms, FL's privacy constraints prohibit such strategies. To tackle CF in HAR environments, we propose FlexFed, a novel FL approach that prioritizes data retention for efficient memory use and dynamically adjusts offline training frequency based on distribution shifts, client capability and offline duration. To better quantify CF in FL, we introduce a new metric that accounts for under-represented data, enabling more accurate evaluations. We also develop a realistic HAR-based evaluation framework that simulates streaming data, dynamic distributions, imbalances and varying availability. Experiments show that FlexFed mitigates CF more effectively, improves FL efficiency by 10 to 15 % and achieves faster, more stable convergence, especially for infrequent or under-represented data.