Neural Networks, Inside Out: Solving for Inputs Given Parameters (A Preliminary Investigation)

TL;DR

This paper explores the possibility of reconstructing training data from observed neural network parameters during training, highlighting potential privacy risks when intermediate parameters are leaked.

Contribution

It introduces the problem of dataset recovery from neural network parameters and provides a preliminary investigation into its feasibility.

Findings

Potential to recover training data from parameter evolution

Highlights privacy vulnerabilities in neural network training

Lays groundwork for further research on data privacy risks

Abstract

Artificial neural network (ANN) is a supervised learning algorithm, where parameters are learned by several back-and-forth iterations of passing the inputs through the network, comparing the output with the expected labels, and correcting the parameters. Inspired by a recent work of Boer and Kramer (2020), we investigate a different problem: Suppose an observer can view how the ANN parameters evolve over many iterations, but the dataset is oblivious to him. For instance, this can be an adversary eavesdropping on a multi-party computation of an ANN parameters (where intermediate parameters are leaked). Can he form a system of equations, and solve it to recover the dataset?