FedHypeVAE: Federated Learning with Hypernetwork Generated Conditional VAEs for Differentially Private Embedding Sharing

TL;DR

FedHypeVAE introduces a privacy-preserving federated generative model using hypernetworks and conditional VAEs, enabling personalized, differentially private embedding sharing across decentralized clients with improved stability under non-IID data.

Contribution

It proposes a novel hypernetwork-driven framework that personalizes generative decoders and ensures differential privacy, addressing limitations of existing embedding generators in federated learning.

Findings

Achieves differential privacy through noise-perturbed gradient aggregation.

Enhances stability and distributional coherence under non-IID data.

Enables controllable multi-domain embedding synthesis.

Abstract

Federated data sharing promises utility without centralizing raw data, yet existing embedding-level generators struggle under non-IID client heterogeneity and provide limited formal protection against gradient leakage. We propose FedHypeVAE, a differentially private, hypernetwork-driven framework for synthesizing embedding-level data across decentralized clients. Building on a conditional VAE backbone, we replace the single global decoder and fixed latent prior with client-aware decoders and class-conditional priors generated by a shared hypernetwork from private, trainable client codes. This bi-level design personalizes the generative layerrather than the downstream modelwhile decoupling local data from communicated parameters. The shared hypernetwork is optimized under differential privacy, ensuring that only noise-perturbed, clipped gradients are aggregated across clients. A local MMD alignment between real and synthetic embeddings and a Lipschitz regularizer on hypernetwork outputs further enhance stability and distributional coherence under non-IID conditions. After training, a neutral meta-code enables domain agnostic synthesis, while mixtures of meta-codes provide controllable multi-domain coverage. FedHypeVAE unifies personalization, privacy, and distribution alignment at the generator level, establishing a principled foundation for privacy-preserving data synthesis in federated settings. Code: github.com/sunnyinAI/FedHypeVAE