Big Data Intelligence Using Distributed Deep Neural Networks

TL;DR

This paper introduces a privacy-preserving distributed deep neural network training algorithm that enables multiple data sources to collaboratively train models without sharing raw data, addressing privacy and resource challenges in sensitive fields.

Contribution

It proposes an improved distributed training algorithm for deep neural networks that maintains data privacy by sharing only model representations, not raw data.

Findings

Comparable performance to centralized training on healthcare data

Effective privacy preservation during distributed training

Facilitates neural network training in privacy-sensitive environments

Abstract

Large amount of data is often required to train and deploy useful machine learning models in industry. Smaller enterprises do not have the luxury of accessing enough data for machine learning, For privacy sensitive fields such as banking, insurance and healthcare, aggregating data to a data warehouse poses a challenge of data security and limited computational resources. These challenges are critical when developing machine learning algorithms in industry. Several attempts have been made to address the above challenges by using distributed learning techniques such as federated learning over disparate data stores in order to circumvent the need for centralised data aggregation. This paper proposes an improved algorithm to securely train deep neural networks over several data sources in a distributed way, in order to eliminate the need to centrally aggregate the data and the need to share the data thus preserving privacy. The proposed method allows training of deep neural networks using data from multiple de-linked nodes in a distributed environment and to secure the representation shared during training. Only a representation of the trained models (network architecture and weights) are shared. The algorithm was evaluated on existing healthcare patients data and the performance of this implementation was compared to that of a regular deep neural network trained on a single centralised architecture. This algorithm will pave a way for distributed training of neural networks on privacy sensitive applications where raw data may not be shared directly or centrally aggregating this data in a data warehouse is not feasible.