Federated Learning for Non-factorizable Models using Deep Generative Prior Approximations

TL;DR

This paper introduces the SIGMA prior, a deep generative model that enables federated learning for non-factorizable models with dependencies, expanding FL's applicability to spatial and dependent data domains.

Contribution

The paper proposes the SIGMA prior, a novel deep generative approach that allows federated learning with models capturing complex dependencies, overcoming previous independence assumptions.

Findings

Effective on synthetic data

Demonstrated in spatial data example

Enables FL for dependent models

Abstract

Federated learning (FL) allows for collaborative model training across decentralized clients while preserving privacy by avoiding data sharing. However, current FL methods assume conditional independence between client models, limiting the use of priors that capture dependence, such as Gaussian processes (GPs). We introduce the Structured Independence via deep Generative Model Approximation (SIGMA) prior which enables FL for non-factorizable models across clients, expanding the applicability of FL to fields such as spatial statistics, epidemiology, environmental science, and other domains where modeling dependencies is crucial. The SIGMA prior is a pre-trained deep generative model that approximates the desired prior and induces a specified conditional independence structure in the latent variables, creating an approximate model suitable for FL settings. We demonstrate the SIGMA prior's effectiveness on synthetic data and showcase its utility in a real-world example of FL for spatial data, using a conditional autoregressive prior to model spatial dependence across Australia. Our work enables new FL applications in domains where modeling dependent data is essential for accurate predictions and decision-making.