On the Reconstruction of Training Data from Group Invariant Networks

TL;DR

This paper investigates the challenge of reconstructing training data from group-invariant neural networks, revealing limitations of existing methods and proposing new approaches with promising initial results.

Contribution

It formulates the problem of data reconstruction from group-invariant networks, evaluates existing techniques, and introduces two novel methods to improve reconstruction quality.

Findings

Conventional reconstruction techniques are ineffective for group-invariant networks.

Reconstructed data tends to be symmetric inputs where the group acts trivially.

Preliminary experiments show potential for the proposed new methods.

Abstract

Reconstructing training data from trained neural networks is an active area of research with significant implications for privacy and explainability. Recent advances have demonstrated the feasibility of this process for several data types. However, reconstructing data from group-invariant neural networks poses distinct challenges that remain largely unexplored. This paper addresses this gap by first formulating the problem and discussing some of its basic properties. We then provide an experimental evaluation demonstrating that conventional reconstruction techniques are inadequate in this scenario. Specifically, we observe that the resulting data reconstructions gravitate toward symmetric inputs on which the group acts trivially, leading to poor-quality results. Finally, we propose two novel methods aiming to improve reconstruction in this setup and present promising preliminary experimental results. Our work sheds light on the complexities of reconstructing data from group invariant neural networks and offers potential avenues for future research in this domain.