FederatedFactory: Generative One-Shot Learning for Extremely Non-IID Distributed Scenarios

TL;DR

FederatedFactory introduces a generative, one-shot federated learning framework that effectively handles highly non-IID data distributions by exchanging generative modules, achieving near-centralized performance without relying on pretrained models.

Contribution

It proposes a novel generative approach to federated learning that replaces traditional weight aggregation, enabling effective learning in extremely non-IID scenarios without external priors.

Findings

Achieves near-centralized performance on medical imaging benchmarks.

Significantly improves accuracy under pathological heterogeneity.

Enables exact modular unlearning of specific generative modules.

Abstract

Federated Learning (FL) enables distributed optimization without compromising data sovereignty. Yet, where local label distributions are mutually exclusive, standard weight aggregation fails due to conflicting optimization trajectories. Often, FL methods rely on pretrained foundation models, introducing unrealistic assumptions. We introduce FederatedFactory, a zero-dependency framework that inverts the unit of federation from discriminative parameters to generative priors. By exchanging generative modules in a single communication round, our architecture supports ex nihilo synthesis of universally class balanced datasets, eliminating gradient conflict and external prior bias entirely. Evaluations across diverse medical imagery benchmarks, including MedMNIST and ISIC2019, demonstrate that our approach recovers centralized upper-bound performance. Under pathological heterogeneity, it lifts baseline accuracy from a collapsed 11.36% to 90.57% on CIFAR-10 and restores ISIC2019 AUROC to 90.57%. Additionally, this framework facilitates exact modular unlearning through the deterministic deletion of specific generative modules.