Towards More Efficient Federated Learning with Better Optimization Objects

TL;DR

This paper proposes a new method for federated learning that uses the aggregation of past models as a constraint to enhance convergence speed and model performance, addressing data heterogeneity issues.

Contribution

It introduces a novel approach of utilizing historical model aggregations as constraints, improving upon existing local training constraints in federated learning.

Findings

Significantly faster convergence in various settings.

Improved model performance compared to baseline algorithms.

Effective handling of data heterogeneity in federated learning.

Abstract

Federated Learning (FL) is a privacy-protected machine learning paradigm that allows model to be trained directly at the edge without uploading data. One of the biggest challenges faced by FL in practical applications is the heterogeneity of edge node data, which will slow down the convergence speed and degrade the performance of the model. For the above problems, a representative solution is to add additional constraints in the local training, such as FedProx, FedCurv and FedCL. However, the above algorithms still have room for improvement. We propose to use the aggregation of all models obtained in the past as new constraint target to further improve the performance of such algorithms. Experiments in various settings demonstrate that our method significantly improves the convergence speed and performance of the model.