Personalized Federated Learning via Stacking

TL;DR

This paper introduces a novel personalized federated learning method using stacking, where clients exchange privacy-preserving models to train a meta-model, improving personalization and contribution assessment across diverse data scenarios.

Contribution

It proposes a flexible stacking-based personalization approach that supports various privacy techniques and federation types, enhancing model customization and contribution evaluation.

Findings

Effective personalization across heterogeneous data scenarios

Supports multiple privacy-preserving techniques and federation types

Enables natural contribution assessment among clients

Abstract

Traditional Federated Learning (FL) methods typically train a single global model collaboratively without exchanging raw data. In contrast, Personalized Federated Learning (PFL) techniques aim to create multiple models that are better tailored to individual clients' data. We present a novel personalization approach based on stacked generalization where clients directly send each other privacy-preserving models to be used as base models to train a meta-model on private data. Our approach is flexible, accommodating various privacy-preserving techniques and model types, and can be applied in horizontal, hybrid, and vertically partitioned federations. Additionally, it offers a natural mechanism for assessing each client's contribution to the federation. Through comprehensive evaluations across diverse simulated data heterogeneity scenarios, we showcase the effectiveness of our method.