Gradient Leakage Attack Resilient Deep Learning

TL;DR

This paper explores resilient deep learning methods against gradient leakage attacks by using dynamic privacy parameters, aiming to improve privacy without sacrificing model accuracy.

Contribution

It introduces adaptive noise strategies for differential privacy in deep learning, enhancing resistance to gradient leakage attacks compared to fixed-parameter methods.

Findings

Dynamic privacy parameters improve attack resistance.

Adaptive noise injection maintains higher model accuracy.

Four metrics for evaluating privacy-preserving approaches.

Abstract

Gradient leakage attacks are considered one of the wickedest privacy threats in deep learning as attackers covertly spy gradient updates during iterative training without compromising model training quality, and yet secretly reconstruct sensitive training data using leaked gradients with high attack success rate. Although deep learning with differential privacy is a defacto standard for publishing deep learning models with differential privacy guarantee, we show that differentially private algorithms with fixed privacy parameters are vulnerable against gradient leakage attacks. This paper investigates alternative approaches to gradient leakage resilient deep learning with differential privacy (DP). First, we analyze existing implementation of deep learning with differential privacy, which use fixed noise variance to injects constant noise to the gradients in all layers using fixed privacy parameters. Despite the DP guarantee provided, the method suffers from low accuracy and is vulnerable to gradient leakage attacks. Second, we present a gradient leakage resilient deep learning approach with differential privacy guarantee by using dynamic privacy parameters. Unlike fixed-parameter strategies that result in constant noise variance, different dynamic parameter strategies present alternative techniques to introduce adaptive noise variance and adaptive noise injection which are closely aligned to the trend of gradient updates during differentially private model training. Finally, we describe four complementary metrics to evaluate and compare alternative approaches.