Federated Learning with Multi-resolution Model Broadcast

TL;DR

This paper introduces a multi-resolution coding scheme for federated learning broadcasts, allowing high-SNR agents to receive more accurate models without extra resources, demonstrated on MNIST.

Contribution

It proposes a novel multi-resolution broadcast method using non-uniform modulation to improve model accuracy for high- and low-SNR agents simultaneously.

Findings

High-SNR agents receive more accurate models.

Low-SNR agents still participate with reduced accuracy.

Effective on MNIST dataset.

Abstract

In federated learning, a server must periodically broadcast a model to the agents. We propose to use multi-resolution coding and modulation (also known as non-uniform modulation) for this purpose. In the simplest instance, broadcast transmission is used, whereby all agents are targeted with one and the same transmission (typically without any particular favored beam direction), which is coded using multi-resolution coding/modulation. This enables high-SNR agents, with high path gains to the server, to receive a more accurate model than the low-SNR agents do, without consuming more downlink resources. As one implementation, we use transmission with a non-uniform 8-PSK constellation, where a high-SNR receiver (agent) can separate all 8 constellation points (hence receive 3 bits) whereas a low-SNR receiver can only separate 4 points (hence receive 2 bits). By encoding the least significant information in the third bit, the high-SNR receivers can obtain the model with higher accuracy, while the low-SNR receiver can still obtain the model although with reduced accuracy, thereby facilitating at least some basic participation of the low-SNR receiver. We show the effectiveness of our proposed scheme via experimentation using federated learning with the MNIST data-set.