Federated Learning on Adaptively Weighted Nodes by Bilevel Optimization

TL;DR

This paper introduces a federated learning approach that adaptively adjusts node weights through bilevel optimization to enhance model performance, offering a communication-efficient algorithm with theoretical guarantees.

Contribution

It presents a novel bilevel optimization framework for adaptively weighting nodes in federated learning, improving performance over static weighting schemes.

Findings

The method achieves better generalization performance under certain conditions.

The proposed algorithm is communication-efficient and theoretically sound.

Scenarios identified where adaptive weighting outperforms static methods.

Abstract

We propose a federated learning method with weighted nodes in which the weights can be modified to optimize the model's performance on a separate validation set. The problem is formulated as a bilevel optimization where the inner problem is a federated learning problem with weighted nodes and the outer problem focuses on optimizing the weights based on the validation performance of the model returned from the inner problem. A communication-efficient federated optimization algorithm is designed to solve this bilevel optimization problem. Under an error-bound assumption, we analyze the generalization performance of the output model and identify scenarios when our method is in theory superior to training a model only locally and to federated learning with static and evenly distributed weights.