Three Tools for Practical Differential Privacy

TL;DR

This paper introduces three practical tools to improve the usability and effectiveness of differentially private machine learning, addressing interpretability, hyperparameter tuning, and privacy testing challenges.

Contribution

The paper presents three novel tools: sanity checks, an adaptive clipping bound, and large-batch training, to enhance practical differential privacy implementation.

Findings

Sanity checks help verify privacy guarantees before training.

Adaptive clipping reduces the need for extensive hyperparameter tuning.

Large-batch training improves model performance under differential privacy.

Abstract

Differentially private learning on real-world data poses challenges for standard machine learning practice: privacy guarantees are difficult to interpret, hyperparameter tuning on private data reduces the privacy budget, and ad-hoc privacy attacks are often required to test model privacy. We introduce three tools to make differentially private machine learning more practical: (1) simple sanity checks which can be carried out in a centralized manner before training, (2) an adaptive clipping bound to reduce the effective number of tuneable privacy parameters, and (3) we show that large-batch training improves model performance.