Client-Conditional Federated Learning via Local Training Data Statistics

TL;DR

This paper introduces a client-conditional federated learning method that uses local PCA statistics to adapt a single global model, improving robustness and performance across diverse data heterogeneity scenarios without extra communication.

Contribution

It proposes a novel approach that conditions a global model on local PCA statistics, eliminating the need for costly clustering or per-client models in heterogeneous federated learning.

Findings

Matches Oracle baseline across all heterogeneity types and datasets.

Surpasses Oracle by 1-6% on combined heterogeneity with richer statistics.

Is uniquely robust to data sparsity among tested methods.

Abstract

Federated learning (FL) under data heterogeneity remains challenging: existing methods either ignore client differences (FedAvg), require costly cluster discovery (IFCA), or maintain per-client models (Ditto). All degrade when data is sparse or heterogeneity is multi-dimensional. We propose conditioning a single global model on locally-computed PCA statistics of each client's training data, requiring zero additional communication. Evaluating across 97~configurations spanning four heterogeneity types (label shift, covariate shift, concept shift, and combined heterogeneity), four datasets (MNIST, Fashion-MNIST, CIFAR-10, CIFAR-100), and seven FL baseline methods, we find that our method matches the Oracle baseline -- which knows true cluster assignments -- across all settings, surpasses it by 1--6% on combined heterogeneity where continuous statistics are richer than discrete cluster identifiers, and is uniquely sparsity-robust among all tested methods.