Deep Unfolding-based Weighted Averaging for Federated Learning in Heterogeneous Environments

TL;DR

This paper introduces a deep unfolding-based method to optimize client aggregation weights in federated learning, improving accuracy and convergence in heterogeneous environments.

Contribution

It presents a novel deep unfolding approach for adaptive weight tuning in federated learning, effectively handling client heterogeneity and enhancing performance.

Findings

Higher test accuracy on class-balanced data.

Effective handling of large-scale models with pretrained support.

Convergence rate analysis provided.

Abstract

Federated learning is a collaborative model training method that iterates model updates by multiple clients and aggregation of the updates by a central server. Device and statistical heterogeneity of participating clients cause significant performance degradation so that an appropriate aggregation weight should be assigned to each client in the aggregation phase of the server. To adjust the aggregation weights, this paper employs deep unfolding, which is known as the parameter tuning method that leverages both learning capability using training data like deep learning and domain knowledge. This enables us to directly incorporate the heterogeneity of the environment of interest into the tuning of the aggregation weights. The proposed approach can be combined with various federated learning algorithms. The results of numerical experiments indicate that a higher test accuracy for unknown class-balanced data can be obtained with the proposed method than that with conventional heuristic weighting methods. The proposed method can handle large-scale learning models with the aid of pretrained models such that it can perform practical real-world tasks. Convergence rate of federated learning algorithms with the proposed method is also provided in this paper.