Decentralized Federated Learning Preserves Model and Data Privacy

TL;DR

This paper introduces a fully decentralized federated learning approach that preserves data and model privacy by sharing knowledge through teacher-student models without transmitting raw data or model parameters.

Contribution

It proposes a novel decentralized knowledge sharing method using teacher-student roles, eliminating the need for a central aggregator and enhancing privacy.

Findings

Untrained student models achieve comparable F1-scores to teachers.

Method enables synchronization of models trained on different data subsets.

No raw data or model parameters are transmitted during training.

Abstract

The increasing complexity of IT systems requires solutions, that support operations in case of failure. Therefore, Artificial Intelligence for System Operations (AIOps) is a field of research that is becoming increasingly focused, both in academia and industry. One of the major issues of this area is the lack of access to adequately labeled data, which is majorly due to legal protection regulations or industrial confidentiality. Methods to mitigate this stir from the area of federated learning, whereby no direct access to training data is required. Original approaches utilize a central instance to perform the model synchronization by periodical aggregation of all model parameters. However, there are many scenarios where trained models cannot be published since its either confidential knowledge or training data could be reconstructed from them. Furthermore the central instance needs to be trusted and is a single point of failure. As a solution, we propose a fully decentralized approach, which allows to share knowledge between trained models. Neither original training data nor model parameters need to be transmitted. The concept relies on teacher and student roles that are assigned to the models, whereby students are trained on the output of their teachers via synthetically generated input data. We conduct a case study on log anomaly detection. The results show that an untrained student model, trained on the teachers output reaches comparable F1-scores as the teacher. In addition, we demonstrate that our method allows the synchronization of several models trained on different distinct training data subsets.