Hierarchical Over-the-Air FedGradNorm

TL;DR

This paper introduces HOTA-FedGradNorm, a hierarchical over-the-air federated learning method with dynamic task weighting that improves training speed and robustness in wireless communication environments.

Contribution

It proposes a novel hierarchical over-the-air federated learning algorithm with dynamic weighting that accounts for channel conditions, enhancing efficiency and robustness.

Findings

Faster training speed compared to static weighting methods.

Improved robustness against channel noise and conditions.

Effective in wireless communication system datasets.

Abstract

Multi-task learning (MTL) is a learning paradigm to learn multiple related tasks simultaneously with a single shared network where each task has a distinct personalized header network for fine-tuning. MTL can be integrated into a federated learning (FL) setting if tasks are distributed across clients and clients have a single shared network, leading to personalized federated learning (PFL). To cope with statistical heterogeneity in the federated setting across clients which can significantly degrade the learning performance, we use a distributed dynamic weighting approach. To perform the communication between the remote parameter server (PS) and the clients efficiently over the noisy channel in a power and bandwidth-limited regime, we utilize over-the-air (OTA) aggregation and hierarchical federated learning (HFL). Thus, we propose hierarchical over-the-air (HOTA) PFL with a dynamic weighting strategy which we call HOTA-FedGradNorm. Our algorithm considers the channel conditions during the dynamic weight selection process. We conduct experiments on a wireless communication system dataset (RadComDynamic). The experimental results demonstrate that the training speed with HOTA-FedGradNorm is faster compared to the algorithms with a naive static equal weighting strategy. In addition, HOTA-FedGradNorm provides robustness against the negative channel effects by compensating for the channel conditions during the dynamic weight selection process.