ALIGN-FL: Architecture-independent Learning through Invariant Generative component sharing in Federated Learning

TL;DR

ALIGN-FL introduces a privacy-preserving federated learning framework that shares generative components instead of full models, enabling effective learning from highly disjoint data distributions across heterogeneous clients.

Contribution

It proposes a novel architecture-independent approach that uses generative component sharing and privacy mechanisms to improve federated learning with non-IID data.

Findings

Effective privacy preservation of outliers in Non-IID scenarios

Successful application on MNIST and Fashion-MNIST datasets

Maintains utility in cross-silo federated learning with heterogeneous clients

Abstract

We present ALIGN-FL, a novel approach to distributed learning that addresses the challenge of learning from highly disjoint data distributions through selective sharing of generative components. Instead of exchanging full model parameters, our framework enables privacy-preserving learning by transferring only generative capabilities across clients, while the server performs global training using synthetic samples. Through complementary privacy mechanisms: DP-SGD with adaptive clipping and Lipschitz regularized VAE decoders and a stateful architecture supporting heterogeneous clients, we experimentally validate our approach on MNIST and Fashion-MNIST datasets with cross-domain outliers. Our analysis demonstrates that both privacy mechanisms effectively map sensitive outliers to typical data points while maintaining utility in extreme Non-IID scenarios typical of cross-silo collaborations. Index Terms: Client-invariant Learning, Federated Learning (FL), Privacy-preserving Generative Models, Non-Independent and Identically Distributed (Non-IID), Heterogeneous Architectures