FedMPQ: Secure and Communication-Efficient Federated Learning with Multi-codebook Product Quantization

TL;DR

FedMPQ introduces a secure, communication-efficient federated learning method using multi-codebook product quantization, significantly reducing uplink data transmission while maintaining high accuracy in non-IID data scenarios.

Contribution

This work presents a novel uplink compression technique for federated learning that enhances security and communication efficiency using multi-codebook product quantization.

Findings

Achieves 99% of baseline accuracy on LEAF dataset.

Reduces uplink communication by 90-95%.

Robust in non-IID and limited public data scenarios.

Abstract

In federated learning, particularly in cross-device scenarios, secure aggregation has recently gained popularity as it effectively defends against inference attacks by malicious aggregators. However, secure aggregation often requires additional communication overhead and can impede the convergence rate of the global model, which is particularly challenging in wireless network environments with extremely limited bandwidth. Therefore, achieving efficient communication compression under the premise of secure aggregation presents a highly challenging and valuable problem. In this work, we propose a novel uplink communication compression method for federated learning, named FedMPQ, which is based on multi shared codebook product quantization.Specifically, we utilize updates from the previous round to generate sufficiently robust codebooks. Secure aggregation is then achieved through trusted execution environments (TEE) or a trusted third party (TTP).In contrast to previous works, our approach exhibits greater robustness in scenarios where data is not independently and identically distributed (non-IID) and there is a lack of sufficient public data. The experiments conducted on the LEAF dataset demonstrate that our proposed method achieves 99% of the baseline's final accuracy, while reducing uplink communications by 90-95%