Deep Learning with Differential Privacy

TL;DR

This paper introduces new algorithms for training deep neural networks with differential privacy, balancing privacy guarantees with model performance and efficiency.

Contribution

It develops refined techniques and analysis for differentially private deep learning, enabling practical training of complex models with privacy protections.

Findings

Successful training of deep neural networks with differential privacy

Maintained model accuracy under modest privacy budgets

Achieved manageable computational costs and software complexity

Abstract

Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains. Often, the training of models requires large, representative datasets, which may be crowdsourced and contain sensitive information. The models should not expose private information in these datasets. Addressing this goal, we develop new algorithmic techniques for learning and a refined analysis of privacy costs within the framework of differential privacy. Our implementation and experiments demonstrate that we can train deep neural networks with non-convex objectives, under a modest privacy budget, and at a manageable cost in software complexity, training efficiency, and model quality.