Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

TL;DR

This paper introduces TCS-H, a novel communication-efficient federated learning method that leverages time-correlated sparsification and hybrid aggregation to improve accuracy under limited wireless communication resources.

Contribution

It proposes a new sparsification technique exploiting temporal correlations and a hybrid aggregation scheme for more efficient over-the-air federated learning.

Findings

TCS-H achieves higher accuracy than traditional methods under limited communication.

The method performs well with both i.i.d. and non-i.i.d. data distributions.

Device scheduling and power allocation further enhance performance.

Abstract

Federated edge learning (FEEL) is a promising distributed machine learning (ML) framework to drive edge intelligence applications. However, due to the dynamic wireless environments and the resource limitations of edge devices, communication becomes a major bottleneck. In this work, we propose time-correlated sparsification with hybrid aggregation (TCS-H) for communication-efficient FEEL, which exploits jointly the power of model compression and over-the-air computation. By exploiting the temporal correlations among model parameters, we construct a global sparsification mask, which is identical across devices, and thus enables efficient model aggregation over-the-air. Each device further constructs a local sparse vector to explore its own important parameters, which are aggregated via digital communication with orthogonal multiple access. We further design device scheduling and power allocation algorithms for TCS-H. Experiment results show that, under limited communication resources, TCS-H can achieve significantly higher accuracy compared to the conventional top-K sparsification with orthogonal model aggregation, with both i.i.d. and non-i.i.d. data distributions.