Efficient Model Compression for Hierarchical Federated Learning

TL;DR

This paper proposes a hierarchical federated learning framework that combines adaptive clustering and model compression techniques to reduce communication costs while maintaining accuracy.

Contribution

It introduces a novel hierarchical FL framework with adaptive clustering and local aggregation with compression, improving communication efficiency in federated learning.

Findings

Significant reduction in energy consumption compared to existing FL methods.

Maintains comparable predictive accuracy with improved communication efficiency.

Effective clustering algorithm for organizing clients in federated learning.

Abstract

Federated learning (FL), as an emerging collaborative learning paradigm, has garnered significant attention due to its capacity to preserve privacy within distributed learning systems. In these systems, clients collaboratively train a unified neural network model using their local datasets and share model parameters rather than raw data, enhancing privacy. Predominantly, FL systems are designed for mobile and edge computing environments where training typically occurs over wireless networks. Consequently, as model sizes increase, the conventional FL frameworks increasingly consume substantial communication resources. To address this challenge and improve communication efficiency, this paper introduces a novel hierarchical FL framework that integrates the benefits of clustered FL and model compression. We present an adaptive clustering algorithm that identifies a core client and dynamically organizes clients into clusters. Furthermore, to enhance transmission efficiency, each core client implements a local aggregation with compression (LC aggregation) algorithm after collecting compressed models from other clients within the same cluster. Simulation results affirm that our proposed algorithms not only maintain comparable predictive accuracy but also significantly reduce energy consumption relative to existing FL mechanisms.