Federated Variational Inference for Bayesian Mixture Models

TL;DR

This paper introduces a federated variational inference method for Bayesian clustering that preserves data privacy while effectively identifying global structures in large-scale binary and categorical datasets.

Contribution

It proposes a novel federated inference procedure combining local and global merge moves using data summaries, enabling privacy-preserving clustering across distributed data sources.

Findings

Performs well compared to existing clustering algorithms on benchmark datasets

Successfully applied to large-scale electronic health record data

Efficiently finds global clustering structures without sharing raw data

Abstract

We present a federated learning approach for Bayesian model-based clustering of large-scale binary and categorical datasets. We introduce a principled 'divide and conquer' inference procedure using variational inference with local merge and delete moves within batches of the data in parallel, followed by 'global' merge moves across batches to find global clustering structures. We show that these merge moves require only summaries of the data in each batch, enabling federated learning across local nodes without requiring the full dataset to be shared. Empirical results on simulated and benchmark datasets demonstrate that our method performs well in comparison to existing clustering algorithms. We validate the practical utility of the method by applying it to large scale electronic health record (EHR) data.