Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing

TL;DR

This paper introduces an over-the-air federated multi-task learning framework that uses model sparsification, compression, and turbo compressed sensing to efficiently handle multiple tasks over shared wireless channels, reducing communication overhead.

Contribution

It proposes a novel over-the-air federated multi-task learning framework with a modified turbo compressed sensing algorithm for concurrent model aggregation and introduces an optimization for power allocation to enhance performance.

Findings

Effectively suppresses inter-task interference.

Achieves learning performance comparable to orthogonal transmission.

Reduces communication overhead through optimized power allocation.

Abstract

To achieve communication-efficient federated multitask learning (FMTL), we propose an over-the-air FMTL (OAFMTL) framework, where multiple learning tasks deployed on edge devices share a non-orthogonal fading channel under the coordination of an edge server (ES). In OA-FMTL, the local updates of edge devices are sparsified, compressed, and then sent over the uplink channel in a superimposed fashion. The ES employs over-the-air computation in the presence of intertask interference. More specifically, the model aggregations of all the tasks are reconstructed from the channel observations concurrently, based on a modified version of the turbo compressed sensing (Turbo-CS) algorithm (named as M-Turbo-CS). We analyze the performance of the proposed OA-FMTL framework together with the M-Turbo-CS algorithm. Furthermore, based on the analysis, we formulate a communication-learning optimization problem to improve the system performance by adjusting the power allocation among the tasks at the edge devices. Numerical simulations show that our proposed OAFMTL effectively suppresses the inter-task interference, and achieves a learning performance comparable to its counterpart with orthogonal multi-task transmission. It is also shown that the proposed inter-task power allocation optimization algorithm substantially reduces the overall communication overhead by appropriately adjusting the power allocation among the tasks.