Intermediate Outputs Are More Sensitive Than You Think

TL;DR

This paper presents a new method for assessing privacy risks in deep computer vision models by analyzing the sensitivity and information content of intermediate outputs without relying on attack simulations.

Contribution

It introduces a novel framework using Degrees of Freedom and Jacobian rank to evaluate privacy risks in model layers, offering a more efficient alternative to existing techniques.

Findings

Effective measurement of privacy risk at different model layers

Deeper insights into information retention in intermediate representations

Framework validated on real-world datasets

Abstract

The increasing reliance on deep computer vision models that process sensitive data has raised significant privacy concerns, particularly regarding the exposure of intermediate results in hidden layers. While traditional privacy risk assessment techniques focus on protecting overall model outputs, they often overlook vulnerabilities within these intermediate representations. Current privacy risk assessment techniques typically rely on specific attack simulations to assess risk, which can be computationally expensive and incomplete. This paper introduces a novel approach to measuring privacy risks in deep computer vision models based on the Degrees of Freedom (DoF) and sensitivity of intermediate outputs, without requiring adversarial attack simulations. We propose a framework that leverages DoF to evaluate the amount of information retained in each layer and combines this with the rank of the Jacobian matrix to assess sensitivity to input variations. This dual analysis enables systematic measurement of privacy risks at various model layers. Our experimental validation on real-world datasets demonstrates the effectiveness of this approach in providing deeper insights into privacy risks associated with intermediate representations.