Utilizing Free Clients in Federated Learning for Focused Model Enhancement

TL;DR

This paper introduces FedALIGN, a federated learning algorithm that selectively includes non-priority clients based on model loss similarity, improving convergence speed and accuracy for prioritized clients.

Contribution

The paper proposes FedALIGN, a novel client selection strategy in federated learning that aligns non-priority clients with global needs, enhancing efficiency and model performance.

Findings

Faster convergence compared to baseline methods

Higher test accuracy on synthetic and benchmark datasets

Effective client selection based on loss similarity

Abstract

Federated Learning (FL) is a distributed machine learning approach to learn models on decentralized heterogeneous data, without the need for clients to share their data. Many existing FL approaches assume that all clients have equal importance and construct a global objective based on all clients. We consider a version of FL we call Prioritized FL, where the goal is to learn a weighted mean objective of a subset of clients, designated as priority clients. An important question arises: How do we choose and incentivize well aligned non priority clients to participate in the federation, while discarding misaligned clients? We present FedALIGN (Federated Adaptive Learning with Inclusion of Global Needs) to address this challenge. The algorithm employs a matching strategy that chooses non priority clients based on how similar the models loss is on their data compared to the global data, thereby ensuring the use of non priority client gradients only when it is beneficial for priority clients. This approach ensures mutual benefits as non priority clients are motivated to join when the model performs satisfactorily on their data, and priority clients can utilize their updates and computational resources when their goals align. We present a convergence analysis that quantifies the trade off between client selection and speed of convergence. Our algorithm shows faster convergence and higher test accuracy than baselines for various synthetic and benchmark datasets.