Federated Learning with Hypergradient-based Online Update of Aggregation Weights

TL;DR

This paper introduces FedHAW, a federated learning method that adaptively updates aggregation weights using hypergradients, enhancing performance in heterogeneous and communication-challenged environments.

Contribution

The paper presents a novel hypergradient-based online update mechanism for aggregation weights in federated learning, improving adaptability and robustness.

Findings

FedHAW achieves high generalization in heterogeneous environments.

The method demonstrates robustness to communication errors.

Low computational overhead for hypergradient calculations.

Abstract

Federated learning using mobile and Internet of Things devices requires not only the ability to handle heterogeneity of clients' data distributions but also high adaptability to varying communication environments. We propose FedHAW (Federated Learning with Hypergradient-based update of Aggregation Weights) that implements online updates of aggregation weights. FedHAW updates the aggregation weights by using hypergradient, the gradient of the objective function with respect to the weights, which can be calculated with low computational overhead. Simulation results show that the proposed method possesses high generalization performance in heterogeneous environments and high robustness to communication errors.