Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learning

TL;DR

This paper introduces Terraform, a deterministic client selection method for federated learning that leverages gradient updates to better handle client heterogeneity, significantly improving model accuracy and efficiency.

Contribution

Terraform is a novel client selection methodology that uses gradient updates and a deterministic algorithm to effectively address heterogeneity in federated learning.

Findings

Achieves up to 47% higher accuracy than prior methods.

Demonstrates robustness through ablation studies and training time analysis.

Effectively selects heterogeneous clients for improved model training.

Abstract

Federated Learning (FL) enables a distributed client-server architecture where multiple clients collaboratively train a global Machine Learning (ML) model without sharing sensitive local data. However, FL often results in lower accuracy than traditional ML algorithms due to statistical heterogeneity across clients. Prior works attempt to address this by using model updates, such as loss and bias, from client models to select participants that can improve the global model's accuracy. However, these updates neither accurately represent a client's heterogeneity nor are their selection methods deterministic. We mitigate these limitations by introducing Terraform, a novel client selection methodology that uses gradient updates and a deterministic selection algorithm to select heterogeneous clients for retraining. This bi-pronged approach allows Terraform to achieve up to 47 percent higher accuracy over prior works. We further demonstrate its efficiency through comprehensive ablation studies and training time analyses, providing strong justification for the robustness of Terraform.