Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature

TL;DR

This paper investigates how the curvature of loss with respect to input data in deep neural networks can be used to improve privacy attacks, particularly membership inference, and provides a theoretical framework for understanding train-test distinguishability based on this curvature.

Contribution

It introduces a novel theoretical framework linking input loss curvature to privacy, and develops a new black box membership inference attack leveraging this curvature, outperforming existing methods.

Findings

Input loss curvature varies between train and test sets.

Curvature-based attack surpasses existing methods on large datasets.

Theoretical bounds relate privacy, dataset size, and distinguishability.

Abstract

In this paper, we explore the properties of loss curvature with respect to input data in deep neural networks. Curvature of loss with respect to input (termed input loss curvature) is the trace of the Hessian of the loss with respect to the input. We investigate how input loss curvature varies between train and test sets, and its implications for train-test distinguishability. We develop a theoretical framework that derives an upper bound on the train-test distinguishability based on privacy and the size of the training set. This novel insight fuels the development of a new black box membership inference attack utilizing input loss curvature. We validate our theoretical findings through experiments in computer vision classification tasks, demonstrating that input loss curvature surpasses existing methods in membership inference effectiveness. Our analysis highlights how the performance of membership inference attack (MIA) methods varies with the size of the training set, showing that curvature-based MIA outperforms other methods on sufficiently large datasets. This condition is often met by real datasets, as demonstrated by our results on CIFAR10, CIFAR100, and ImageNet. These findings not only advance our understanding of deep neural network behavior but also improve the ability to test privacy-preserving techniques in machine learning.