Communication-Efficient Diffusion Strategy for Performance Improvement of Federated Learning with Non-IID Data

TL;DR

This paper introduces FedDif, a diffusion strategy for federated learning with non-IID data, enhancing global model performance and communication efficiency through device-to-device model passing and auction-based trade-offs.

Contribution

The paper proposes FedDif, a novel diffusion approach that addresses weight divergence in non-IID federated learning and optimizes communication efficiency using auction theory.

Findings

FedDif improves top-1 accuracy by up to 34.89%.

It reduces communication costs by up to 63.49%.

Theoretical analysis confirms FedDif's ability to circumvent weight divergence.

Abstract

In 6G mobile communication systems, various AI-based network functions and applications have been standardized. Federated learning (FL) is adopted as the core learning architecture for 6G systems to avoid privacy leakage from mobile user data. However, in FL, users with non-independent and identically distributed (non-IID) datasets can deteriorate the performance of the global model because the convergence direction of the gradient for each dataset is different, thereby inducing a weight divergence problem. To address this problem, we propose a novel diffusion strategy for machine learning (ML) models (FedDif) to maximize the performance of the global model with non-IID data. FedDif enables the local model to learn different distributions before parameter aggregation by passing the local models through users via device-to-device communication. Furthermore, we theoretically demonstrate that FedDif can circumvent the weight-divergence problem. Based on this theory, we propose a communication-efficient diffusion strategy for ML models that can determine the trade-off between learning performance and communication cost using auction theory. The experimental results show that FedDif improves the top-1 test accuracy by up to 34.89\% and reduces communication costs by 14.6% to a maximum of 63.49%.