Communication-Efficient Federated Learning with Binary Neural Networks

TL;DR

This paper proposes a communication-efficient federated learning framework using binary neural networks, reducing communication overhead while maintaining performance through a novel training scheme and theoretical convergence analysis.

Contribution

It introduces a new federated learning framework for binary neural networks with a maximum likelihood-based update scheme and provides the first theoretical convergence conditions.

Findings

Significantly reduces communication cost compared to real-valued neural networks.

Maintains competitive performance despite binarization.

Hybrid methods can further compensate for performance loss.

Abstract

Federated learning (FL) is a privacy-preserving machine learning setting that enables many devices to jointly train a shared global model without the need to reveal their data to a central server. However, FL involves a frequent exchange of the parameters between all the clients and the server that coordinates the training. This introduces extensive communication overhead, which can be a major bottleneck in FL with limited communication links. In this paper, we consider training the binary neural networks (BNN) in the FL setting instead of the typical real-valued neural networks to fulfill the stringent delay and efficiency requirement in wireless edge networks. We introduce a novel FL framework of training BNN, where the clients only upload the binary parameters to the server. We also propose a novel parameter updating scheme based on the Maximum Likelihood (ML) estimation that preserves the performance of the BNN even without the availability of aggregated real-valued auxiliary parameters that are usually needed during the training of the BNN. Moreover, for the first time in the literature, we theoretically derive the conditions under which the training of BNN is converging. { Numerical results show that the proposed FL framework significantly reduces the communication cost compared to the conventional neural networks with typical real-valued parameters, and the performance loss incurred by the binarization can be further compensated by a hybrid method.