Quantized Rank Reduction: A Communications-Efficient Federated Learning Scheme for Network-Critical Applications

TL;DR

This paper introduces a federated learning scheme that combines low-rank approximation and quantization of neural network gradients to reduce communication costs while maintaining model accuracy in network-critical applications.

Contribution

It presents a novel approach that integrates rank reduction and quantization techniques to enhance communication efficiency in federated learning.

Findings

Significant reduction in communication load achieved

Minimal impact on model accuracy demonstrated

Applicable to network-critical federated learning scenarios

Abstract

Federated learning is a machine learning approach that enables multiple devices (i.e., agents) to train a shared model cooperatively without exchanging raw data. This technique keeps data localized on user devices, ensuring privacy and security, while each agent trains the model on their own data and only shares model updates. The communication overhead is a significant challenge due to the frequent exchange of model updates between the agents and the central server. In this paper, we propose a communication-efficient federated learning scheme that utilizes low-rank approximation of neural network gradients and quantization to significantly reduce the network load of the decentralized learning process with minimal impact on the model's accuracy.