# Differentially Private Model Publishing for Deep Learning

## TL;DR

This paper introduces a differentially private training method for neural networks that enhances privacy guarantees while maintaining model accuracy, using concentrated differential privacy and dynamic privacy budgeting.

## Contribution

It presents a novel differentially private training approach employing concentrated differential privacy and dynamic privacy budget allocation to improve privacy-accuracy trade-offs.

## Key findings

- Effective privacy loss accounting and improved model accuracy.
- Enhanced training efficiency under privacy constraints.
- Maintains model quality with strong privacy guarantees.

## Abstract

Deep learning techniques based on neural networks have shown significant success in a wide range of AI tasks. Large-scale training datasets are one of the critical factors for their success. However, when the training datasets are crowdsourced from individuals and contain sensitive information, the model parameters may encode private information and bear the risks of privacy leakage. The recent growing trend of the sharing and publishing of pre-trained models further aggravates such privacy risks. To tackle this problem, we propose a differentially private approach for training neural networks. Our approach includes several new techniques for optimizing both privacy loss and model accuracy. We employ a generalization of differential privacy called concentrated differential privacy(CDP), with both a formal and refined privacy loss analysis on two different data batching methods. We implement a dynamic privacy budget allocator over the course of training to improve model accuracy. Extensive experiments demonstrate that our approach effectively improves privacy loss accounting, training efficiency and model quality under a given privacy budget.