Data Selection for Efficient Model Update in Federated Learning

TL;DR

This paper introduces a method to improve federated learning efficiency by selecting and transmitting only the most representative data features from clients, significantly reducing communication overhead while maintaining model performance.

Contribution

It proposes a clustering-based data selection approach that minimizes knowledge exchange in federated learning with split neural networks, enhancing communication efficiency.

Findings

Only 1.6% of initial data exchange suffices for effective model updates

Selected data features effectively capture client data characteristics

The approach improves communication efficiency without sacrificing accuracy

Abstract

The Federated Learning (FL) workflow of training a centralized model with distributed data is growing in popularity. However, until recently, this was the realm of contributing clients with similar computing capability. The fast expanding IoT space and data being generated and processed at the edge are encouraging more effort into expanding federated learning to include heterogeneous systems. Previous approaches distribute light-weight models to clients are rely on knowledge transfer to distil the characteristic of local data in partitioned updates. However, their additional knowledge exchange transmitted through the network degrades the communication efficiency of FL. We propose to reduce the size of knowledge exchanged in these FL setups by clustering and selecting only the most representative bits of information from the clients. The partitioned global update adopted in our work splits the global deep neural network into a lower part for generic feature extraction and an upper part that is more sensitive to this selected client knowledge. Our experiments show that only 1.6% of the initially exchanged data can effectively transfer the characteristic of the client data to the global model in our FL approach, using split networks. These preliminary results evolve our understanding of federated learning by demonstrating efficient training using strategically selected training samples.