FedBAT: Communication-Efficient Federated Learning via Learnable Binarization

TL;DR

FedBAT introduces a novel federated learning framework that learns binary model updates during training, significantly reducing communication costs and improving convergence and accuracy over traditional methods.

Contribution

It proposes a learnable binarization-aware training method with theoretical convergence guarantees, enhancing efficiency and accuracy in federated learning.

Findings

FedBAT accelerates convergence compared to baselines.

FedBAT surpasses baseline accuracy by up to 9%.

FedBAT can outperform FedAvg in some cases.

Abstract

Federated learning is a promising distributed machine learning paradigm that can effectively exploit large-scale data without exposing users' privacy. However, it may incur significant communication overhead, thereby potentially impairing the training efficiency. To address this challenge, numerous studies suggest binarizing the model updates. Nonetheless, traditional methods usually binarize model updates in a post-training manner, resulting in significant approximation errors and consequent degradation in model accuracy. To this end, we propose Federated Binarization-Aware Training (FedBAT), a novel framework that directly learns binary model updates during the local training process, thus inherently reducing the approximation errors. FedBAT incorporates an innovative binarization operator, along with meticulously designed derivatives to facilitate efficient learning. In addition, we establish theoretical guarantees regarding the convergence of FedBAT. Extensive experiments are conducted on four popular datasets. The results show that FedBAT significantly accelerates the convergence and exceeds the accuracy of baselines by up to 9\%, even surpassing that of FedAvg in some cases.