Representation Theorem for Matrix Product States

TL;DR

This paper explores the expressive power of Matrix Product States (MPS), demonstrating their ability to represent arbitrary boolean and continuous functions, and establishing connections with neural networks and Gaussian processes.

Contribution

It provides a construction method for MPS representing boolean functions, proves the density of MPS with sigmoidal activation in continuous functions, and links MPS to neural networks and Gaussian processes.

Findings

MPS can accurately realize arbitrary boolean functions.

MPS with sigmoidal activation are dense in continuous functions.

Equivalent neural networks for MPS involve non-linear kernels like polynomial kernels.

Abstract

In this work, we investigate the universal representation capacity of the Matrix Product States (MPS) from the perspective of boolean functions and continuous functions. We show that MPS can accurately realize arbitrary boolean functions by providing a construction method of the corresponding MPS structure for an arbitrarily given boolean gate. Moreover, we prove that the function space of MPS with the scale-invariant sigmoidal activation is dense in the space of continuous functions defined on a compact subspace of the $n$-dimensional real coordinate space $\mathbb{R^{n}}$. We study the relation between MPS and neural networks and show that the MPS with a scale-invariant sigmoidal function is equivalent to a one-hidden-layer neural network equipped with a kernel function. We construct the equivalent neural networks for several specific MPS models and show that non-linear kernels such as the polynomial kernel which introduces the couplings between different components of the input into the model appear naturally in the equivalent neural networks. At last, we discuss the realization of the Gaussian Process (GP) with infinitely wide MPS by studying their equivalent neural networks.