Differential Privacy Regularization: Protecting Training Data Through Loss Function Regularization

TL;DR

This paper introduces a new regularization method for neural network training that aims to protect sensitive data by achieving differential privacy more efficiently than existing methods like DP-SGD.

Contribution

A novel regularization strategy is proposed to enhance differential privacy in neural network training without modifying the standard SGD algorithm.

Findings

The regularization method effectively protects training data privacy.

It offers improved efficiency over DP-SGD.

The approach maintains model performance while ensuring privacy.

Abstract

Training machine learning models based on neural networks requires large datasets, which may contain sensitive information. The models, however, should not expose private information from these datasets. Differentially private SGD [DP-SGD] requires the modification of the standard stochastic gradient descent [SGD] algorithm for training new models. In this short paper, a novel regularization strategy is proposed to achieve the same goal in a more efficient manner.