FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation

TL;DR

FedCache 2.0 advances federated edge learning by integrating knowledge caching and dataset distillation, enabling personalized, efficient, and privacy-preserving model training across diverse devices and data types.

Contribution

It introduces a novel architecture combining dataset distillation and knowledge cache strategies for personalized federated learning with improved communication efficiency.

Findings

Outperforms state-of-the-art methods across multiple datasets and modalities.

Achieves at least 28.6 times improvement in communication efficiency.

Effectively trains personalized on-device models in heterogeneous environments.

Abstract

Federated Edge Learning (FEL) has emerged as a promising approach for enabling edge devices to collaboratively train machine learning models while preserving data privacy. Despite its advantages, practical FEL deployment faces significant challenges related to device constraints and device-server interactions, necessitating heterogeneous, user-adaptive model training with limited and uncertain communication. In this paper, we introduce FedCache 2.0, a novel personalized FEL architecture that simultaneously addresses these challenges. FedCache 2.0 incorporates the benefits of both dataset distillation and knowledge cache-driven federated learning by storing and organizing distilled data as knowledge in the server-side knowledge cache. Moreover, a device-centric cache sampling strategy is introduced to tailor transferred knowledge for individual devices within controlled communication bandwidth. Extensive experiments on five datasets covering image recognition, audio understanding, and mobile sensor data mining tasks demonstrate that (1) FedCache 2.0 significantly outperforms state-of-the-art methods regardless of model structures, data distributions, and modalities. (2) FedCache 2.0 can train splendid personalized on-device models with at least $\times$28.6 improvement in communication efficiency.