PriPHiT: Privacy-Preserving Hierarchical Training of Deep Neural Networks

TL;DR

PriPHiT enables privacy-preserving training of deep neural networks by combining edge-based adversarial early exits with differential privacy, reducing sensitive data transmission to the cloud while maintaining high model performance.

Contribution

It introduces a novel hierarchical training method that preserves privacy on resource-constrained devices using adversarial early exits and noise addition for differential privacy.

Findings

Effective privacy preservation against reconstruction attacks

High accuracy on facial and medical datasets

Low computational overhead suitable for edge devices

Abstract

The training phase of deep neural networks requires substantial resources and as such is often performed on cloud servers. However, this raises privacy concerns when the training dataset contains sensitive content, e.g., facial or medical images. In this work, we propose a method to perform the training phase of a deep learning model on both an edge device and a cloud server that prevents sensitive content being transmitted to the cloud while retaining the desired information. The proposed privacy-preserving method uses adversarial early exits to suppress the sensitive content at the edge and transmits the task-relevant information to the cloud. This approach incorporates noise addition during the training phase to provide a differential privacy guarantee. We extensively test our method on different facial and medical datasets with diverse attributes using various deep learning architectures, showcasing its outstanding performance. We also demonstrate the effectiveness of privacy preservation through successful defenses against different white-box, deep and GAN-based reconstruction attacks. This approach is designed for resource-constrained edge devices, ensuring minimal memory usage and computational overhead.