PATROL: Privacy-Oriented Pruning for Collaborative Inference Against Model Inversion Attacks

TL;DR

PATROL is a privacy-preserving pruning method for collaborative inference that enhances model robustness against inversion attacks while maintaining inference accuracy, by selectively deploying layers at the edge.

Contribution

It introduces a novel privacy-oriented pruning technique combining Lipschitz regularization and adversarial training to defend against model inversion attacks in collaborative inference.

Findings

PATROL significantly reduces the success rate of model inversion attacks.

It maintains high inference accuracy with limited edge resources.

Demonstrated effectiveness on vehicle re-identification tasks.

Abstract

Collaborative inference has been a promising solution to enable resource-constrained edge devices to perform inference using state-of-the-art deep neural networks (DNNs). In collaborative inference, the edge device first feeds the input to a partial DNN locally and then uploads the intermediate result to the cloud to complete the inference. However, recent research indicates model inversion attacks (MIAs) can reconstruct input data from intermediate results, posing serious privacy concerns for collaborative inference. Existing perturbation and cryptography techniques are inefficient and unreliable in defending against MIAs while performing accurate inference. This paper provides a viable solution, named PATROL, which develops privacy-oriented pruning to balance privacy, efficiency, and utility of collaborative inference. PATROL takes advantage of the fact that later layers in a DNN can extract more task-specific features. Given limited local resources for collaborative inference, PATROL intends to deploy more layers at the edge based on pruning techniques to enforce task-specific features for inference and reduce task-irrelevant but sensitive features for privacy preservation. To achieve privacy-oriented pruning, PATROL introduces two key components: Lipschitz regularization and adversarial reconstruction training, which increase the reconstruction errors by reducing the stability of MIAs and enhance the target inference model by adversarial training, respectively. On a real-world collaborative inference task, vehicle re-identification, we demonstrate the superior performance of PATROL in terms of against MIAs.