NeuralDP Differentially private neural networks by design

TL;DR

NeuralDP introduces a layer activation privatization method for neural networks that improves privacy-utility trade-offs over traditional DP-SGD, enabling more effective privacy-preserving deep learning.

Contribution

The paper proposes NeuralDP, a novel approach for differential privacy in neural networks by privatizing layer activations, offering better privacy-utility trade-offs.

Findings

Outperforms DP-SGD on MNIST and PPD datasets

Provides stronger privacy guarantees with improved utility

Demonstrates effectiveness of activation privatization approach

Abstract

The application of differential privacy to the training of deep neural networks holds the promise of allowing large-scale (decentralized) use of sensitive data while providing rigorous privacy guarantees to the individual. The predominant approach to differentially private training of neural networks is DP-SGD, which relies on norm-based gradient clipping as a method for bounding sensitivity, followed by the addition of appropriately calibrated Gaussian noise. In this work we propose NeuralDP, a technique for privatising activations of some layer within a neural network, which by the post-processing properties of differential privacy yields a differentially private network. We experimentally demonstrate on two datasets (MNIST and Pediatric Pneumonia Dataset (PPD)) that our method offers substantially improved privacy-utility trade-offs compared to DP-SGD.