Global Update Guided Federated Learning

TL;DR

This paper introduces FedGG, a federated learning method that uses global-update guidance and adaptive loss weights to improve convergence and accuracy in the presence of unbalanced data distributions.

Contribution

The paper proposes a novel federated learning approach that incorporates a model-cosine loss and adaptive weighting to enhance training stability and performance.

Findings

FedGG outperforms existing algorithms in convergence speed and accuracy.

Adaptive loss weights improve stability and ease of implementation.

Numerical simulations validate the effectiveness of FedGG.

Abstract

Federated learning protects data privacy and security by exchanging models instead of data. However, unbalanced data distributions among participating clients compromise the accuracy and convergence speed of federated learning algorithms. To alleviate this problem, unlike previous studies that limit the distance of updates for local models, we propose global-update-guided federated learning (FedGG), which introduces a model-cosine loss into local objective functions, so that local models can fit local data distributions under the guidance of update directions of global models. Furthermore, considering that the update direction of a global model is informative in the early stage of training, we propose adaptive loss weights based on the update distances of local models. Numerical simulations show that, compared with other advanced algorithms, FedGG has a significant improvement on model convergence accuracies and speeds. Additionally, compared with traditional fixed loss weights, adaptive loss weights enable our algorithm to be more stable and easier to implement in practice.