Federated Neural Topic Models

TL;DR

This paper introduces a federated neural topic modeling approach that enables multiple parties to collaboratively train a neural topic model without sharing their data, maintaining privacy while achieving results similar to centralized training.

Contribution

It is the first to adapt neural topic models to a federated setting, demonstrating benefits in privacy preservation and handling diverse topic distributions across nodes.

Findings

Federated neural topic models perform comparably to centralized models.

The approach preserves data privacy across nodes.

Experiments show effectiveness on synthetic and real datasets.

Abstract

Over the last years, topic modeling has emerged as a powerful technique for organizing and summarizing big collections of documents or searching for particular patterns in them. However, privacy concerns may arise when cross-analyzing data from different sources. Federated topic modeling solves this issue by allowing multiple parties to jointly train a topic model without sharing their data. While several federated approximations of classical topic models do exist, no research has been conducted on their application for neural topic models. To fill this gap, we propose and analyze a federated implementation based on state-of-the-art neural topic modeling implementations, showing its benefits when there is a diversity of topics across the nodes' documents and the need to build a joint model. In practice, our approach is equivalent to a centralized model training, but preserves the privacy of the nodes. Advantages of this federated scenario are illustrated by means of experiments using both synthetic and real data scenarios.