FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learning

TL;DR

FedCor introduces a correlation-based client selection strategy using Gaussian Processes to significantly improve convergence rates in heterogeneous federated learning, addressing data heterogeneity and client loss correlations.

Contribution

The paper proposes FedCor, a novel FL framework that models client loss correlations with Gaussian Processes to optimize client selection and accelerate convergence.

Findings

FedCor improves convergence rates by 34% to 99% on FMNIST.

FedCor improves convergence rates by 26% to 51% on CIFAR-10.

Efficient GP training reduces communication overhead in FL.

Abstract

Client-wise data heterogeneity is one of the major issues that hinder effective training in federated learning (FL). Since the data distribution on each client may vary dramatically, the client selection strategy can significantly influence the convergence rate of the FL process. Active client selection strategies are popularly proposed in recent studies. However, they neglect the loss correlations between the clients and achieve only marginal improvement compared to the uniform selection strategy. In this work, we propose FedCor -- an FL framework built on a correlation-based client selection strategy, to boost the convergence rate of FL. Specifically, we first model the loss correlations between the clients with a Gaussian Process (GP). Based on the GP model, we derive a client selection strategy with a significant reduction of expected global loss in each round. Besides, we develop an efficient GP training method with a low communication overhead in the FL scenario by utilizing the covariance stationarity. Our experimental results show that compared to the state-of-the-art method, FedCorr can improve the convergence rates by $34\%\sim 99\%$ and $26\%\sim 51\%$ on FMNIST and CIFAR-10, respectively.