Tensorization of neural networks for improved privacy and interpretability

TL;DR

This paper introduces a tensorization algorithm for neural networks that enhances privacy and interpretability by constructing tensor train representations using black-box access, with applications in data obfuscation, topological phase estimation, and model compression.

Contribution

The paper presents a novel tensorization method for neural networks that improves privacy, interpretability, and efficiency, applicable with minimal data and offering advantages over traditional tensorization techniques.

Findings

Effective obfuscation of neural network parameters.

Accurate estimation of topological phases from MPS.

Superior memory-time trade-offs in model compression.

Abstract

We present a tensorization algorithm for constructing tensor train/matrix product state (MPS) representations of functions, drawing on sketching and cross interpolation ideas. The method only requires black-box access to the target function and a small set of sample points defining the domain of interest. Thus, it is particularly well-suited for machine learning models, where the domain of interest is naturally defined by the training dataset. We show that this approach can be used to enhance the privacy and interpretability of neural network models. Specifically, we apply our decomposition to (i) obfuscate neural networks whose parameters encode patterns tied to the training data distribution, and (ii) estimate topological phases of matter that are easily accessible from the MPS representation. Additionally, we show that this tensorization can serve as an efficient initialization method for optimizing MPS in general settings, and that, for model compression, our algorithm achieves a superior trade-off between memory and time complexity compared to conventional tensorization methods of neural networks.