FedDES: Graph-Based Dynamic Ensemble Selection for Personalized Federated Learning

TL;DR

FedDES introduces a graph-based dynamic ensemble selection method for personalized federated learning, enabling instance-level model customization and improving performance in heterogeneous data environments.

Contribution

The paper proposes FedDES, a novel decentralized pFL framework using GNNs for dynamic peer model selection at the instance level, addressing limitations of uniform peer contribution.

Findings

FedDES outperforms existing pFL methods on CIFAR-10 and ICU healthcare data.

It effectively suppresses irrelevant peer models, reducing negative transfer.

Demonstrates robustness in non-IID data settings.

Abstract

Statistical heterogeneity in Federated Learning (FL) often leads to negative transfer, where a single global model fails to serve diverse client distributions. Personalized federated learning (pFL) aims to address this by tailoring models to individual clients. However, under most existing pFL approaches, clients integrate peer client contributions uniformly, which ignores the reality that not all peers are likely to be equally beneficial. Additionally, the potential for personalization at the instance level remains largely unexplored, even though the reliability of different peer models often varies across individual samples within the same client. We introduce FedDES (Federated Dynamic Ensemble Selection), a decentralized pFL framework that achieves instance-level personalization through dynamic ensemble selection. Central to our approach is a Graph Neural Network (GNN) meta-learner trained on a heterogeneous graph modeling interactions between data samples and candidate classifiers. For each test query, the GNN dynamically selects and weights peer client models, forming an ensemble of the most competent classifiers while effectively suppressing contributions from those that are irrelevant or potentially harmful for performance. Experiments on CIFAR-10 and real-world ICU healthcare data demonstrate that FedDES outperforms state-of-the-art pFL baselines in non-IID settings, offering robust protection against negative transfer.