Neural network representation of tensor network and chiral states

TL;DR

This paper demonstrates that Boltzmann machines can efficiently represent complex quantum many-body states, including tensor network states and certain chiral topological states, highlighting their expressive power.

Contribution

It proves that local tensor network states can be represented by neural networks with near-linear parameter count and constructs a neural network for chiral p-wave superconductors.

Findings

Neural networks can represent any local tensor network state.

Constructed a neural network for chiral p-wave superconductor.

Representation is nearly optimal in parameter efficiency.

Abstract

We study the representational power of Boltzmann machines (a type of neural network) in quantum many-body systems. We prove that any (local) tensor network state has a (local) neural network representation. The construction is almost optimal in the sense that the number of parameters in the neural network representation is almost linear in the number of nonzero parameters in the tensor network representation. Despite the difficulty of representing (gapped) chiral topological states with local tensor networks, we construct a quasi-local neural network representation for a chiral $p$-wave superconductor. These results demonstrate the power of Boltzmann machines.

Equations

$$E(\{s_j\})=\sum _j{h}_j{s}_j+\sum _{j,k}{w}_{jk}{s}_j{s}_k.$$

$$Z=\sum _{\{s_j\}\in \{\pm 1{\}}^{\times (|V|+|H|)}}{e}^{-E(\{s_j\})}$$

$$P\{s_{j\le |V|}\}=\frac{1}{Z}\sum _{\{s_{j>|V|}\}\in \{\pm 1{\}}^{\times |H|}}{e}^{-E(\{s_j\})}.$$

$$|\psi ⟩=\sum _{\{s_1,s_2,\dots ,s_{|V|+|H|}\}\in \{\pm 1{\}}^{\times (|V|+|H|)}}{e}^{-{\sum }_j{h}_j{s}_j-{\sum }_{j,k}{w}_{jk}{s}_j{s}_k}|\{s_1,s_2,\dots ,s_{|V|}\}⟩.$$

$$|\psi ⟩=\sum _{\{s_j\}\in \{\pm 1{\}}^{\times l^{\prime }}}{T}^{\prime }\left(\frac{s_1+3}{2},\frac{s_2+3}{2},\dots ,\frac{s_n+3}{2}\right)|\{s_j\}⟩.$$

$$\sum _{j=1}^{d_2}{T}_1(j_1,j,j_3)T_2(j_1^{\prime },j,j_3^{\prime }).$$

$$\sum _{\{s_1,s_2,\dots ,s_{|V|+|H|}\}\in \{\pm 1{\}}^{\times (|V|+|H|)},\{s_1^{\prime },s_2^{\prime },\dots ,s_{|V^{\prime }|+|H^{\prime }|}^{\prime }\}\in \{\pm 1{\}}^{\times (|V^{\prime }|+|H^{\prime }|)}\text{s.t.}{s}_1=s_1^{\prime },s_2=s_2^{\prime },\dots ,s_c=s_c^{\prime }}\cdots ,$$

$$e^{-{\sum }_j{h}_j{s}_j-{\sum }_{j,k}{w}_{jk}{s}_j{s}_k-{\sum }_j{h}_j^{\prime }{s}_j^{\prime }-{\sum }_{j,k}{w}_{jk}^{\prime }{s}_j^{\prime }{s}_k^{\prime }}|\{s_{c+1},s_{c+2},\dots ,s_{|V|}\}⟩\otimes |\{s_{c+1}^{\prime },s_{c+2}^{\prime },\dots ,s_{|V^{\prime }|}^{\prime }\}⟩.$$

$$|H|=\sum _{j=1}^n(\varnothing (T_j)+{\log }_2\#(T_j)+O(1)),$$

$$\sum _j\varnothing (T_j){\log }_2\#(T_j)+\text{less significant terms}.$$

$$|\psi ⟩=\left(\int e^{-{\sum }_{j,k}{w}_{jk}{\xi }_j{\xi }_k}\prod _{l=1}^{|H|}\mathrm{d}{\xi }_{|V|+l}\right)|0⟩$$

$$H=\sum _{\vec{x}\in {\mathbb{Z}}^2}{c}_{\vec{x}+\vec{i}}^{†}{c}_{\vec{x}}+c_{\vec{x}+\vec{j}}^{†}{c}_{\vec{x}}+c_{\vec{x}+\vec{i}}^{†}{c}_{\vec{x}}^{†}+i{c}_{\vec{x}+\vec{j}}^{†}{c}_{\vec{x}}^{†}+H.c.-2\mu \sum _{\vec{x}\in {\mathbb{Z}}^2}{c}_{\vec{x}}^{†}{c}_{\vec{x}},\mu \in \mathbb{R}.$$

$$c_{\vec{x}}^{†}={\int }_{\text{BZ}}{e}^{-i\vec{k}\cdot \vec{x}}{c}_{\vec{k}}^{†}\mathrm{d}\vec{k},c_{\vec{x}}={\int }_{\text{BZ}}{e}^{i\vec{k}\cdot \vec{x}}{c}_{\vec{k}}\mathrm{d}\vec{k}$$

$$H={\int }_{\text{BZ}}{H}_{\vec{k}}\mathrm{d}\vec{k},H_{\vec{k}}={\Delta }_{\vec{k}}{c}_{\vec{k}}^{†}{c}_{-\vec{k}}^{†}+{\Delta }_{\vec{k}}^{\ast }{c}_{-\vec{k}}{c}_{\vec{k}}+2M_{\vec{k}}{c}_{\vec{k}}^{†}{c}_{\vec{k}}$$

$${\Delta }_{\vec{k}}=\sin k_y-i\sin k_x,M_{\vec{k}}=\cos k_x+\cos k_y-\mu .$$

$$E_{\vec{k}}=2\sqrt{|{\Delta }_{\vec{k}}{|}^2+M_{\vec{k}}^2}=2\sqrt{{\sin }^2{k}_x+{\sin }^2{k}_y+(\cos k_x+\cos k_y-\mu {)}^2}.$$

$$|\psi ⟩\propto e^{\frac{1}{2}{\int }_{\text{BZ}}\frac{v_{\vec{k}}}{u_{\vec{k}}}{c}_{\vec{k}}^{†}{c}_{-\vec{k}}^{†}\mathrm{d}\vec{k}}|0⟩,$$

$$u_{\vec{k}}=-\sqrt{|{\Delta }_{\vec{k}}{|}^2+M_{\vec{k}}^2}-M_{\vec{k}},v_{\vec{k}}={\Delta }_{\vec{k}}$$

$${\xi }_{\vec{x}}={\int }_{\text{B}Z}{e}^{-i\vec{k}\cdot \vec{x}}{\xi }_{\vec{k}}\mathrm{d}\vec{k}.$$

$$|\psi ⟩\propto \left(\int [\mathrm{d}{\xi }_{\vec{k}}]e^{{\int }_{\text{BZ}}(v_{\vec{k}}{\xi }_{-\vec{k}}{c}_{\vec{k}}^{†}-\frac{1}{2}{\xi }_{-\vec{k}}{v}_{-\vec{k}}{u}_{\vec{k}}{\xi }_{\vec{k}})\mathrm{d}\vec{k}}\right)|0⟩,$$

$$\text{hBZ}=\{\vec{k}\in \text{BZ}|k_x>0\vee (k_x=0\wedge k_y>0)\}$$

$$c_{\vec{x}}^{†}=\frac{1}{\sqrt{L}}\sum _{\vec{k}\in \mathrm{BZ}}{e}^{-i\vec{k}\cdot \vec{x}}{c}_{\vec{k}}^{†},c_{\vec{x}}=\frac{1}{\sqrt{L}}\sum _{\vec{k}\in \mathrm{BZ}}{e}^{i\vec{k}\cdot \vec{x}}{c}_{\vec{k}},$$

$$H=2\sum _{\vec{k}\in \text{hBZ}}(M_{\vec{k}}{c}_{\vec{k}}^{†}{c}_{\vec{k}}+M_{\vec{k}}{c}_{-\vec{k}}^{†}{c}_{-\vec{k}}+{\Delta }_{\vec{k}}{c}_{\vec{k}}^{†}{c}_{-\vec{k}}^{†}+{\Delta }_{\vec{k}}^{\ast }{c}_{-\vec{k}}{c}_{\vec{k}})+2M_{\vec{k}=\vec{0}}{c}_{\vec{k}=\vec{0}}^{†}{c}_{\vec{k}=\vec{0}}.$$

$$|\psi ⟩=\underset{\vec{k}\in \text{hBZ}}{⨂}|{\psi }_{\vec{k}}⟩,|{\psi }_{\vec{k}}⟩\propto e^{v_{\vec{k}}{c}_{\vec{k}}^{†}{c}_{-\vec{k}}^{†}/u_{\vec{k}}}|0⟩.$$

$$|{\varphi }^{(m)}⟩=\underset{\vec{k}\in \text{hBZ}}{⨂}|{\varphi }_{\vec{k}}^{(m)}⟩,|{\varphi }_{\vec{k}}^{(m)}⟩\propto e^{v_{\vec{k}}{c}_{\vec{k}}^{†}{c}_{-\vec{k}}^{†}/u_{\vec{k}}^{(m)}}|0⟩,$$

$$|u_{\vec{k}}^{(m)}-u_{\vec{k}}|=e^{-\Omega (ϵm)},\forall \vec{k}\in \text{hBZ}.$$

$$|u_{\vec{k}}|\ge \Omega (L^{-2}),|v_{\vec{k}}|\ge \Omega (L^{-1}),\forall \vec{k}\in \text{hBZ}.$$

$$f_{\vec{k}}:=|⟨{\varphi }_{\vec{k}}^{(m)}|{\psi }_{\vec{k}}⟩|\ge 1-1/\text{poly}(L),\forall \vec{k}\in \text{hBZ}$$

$$|⟨{\varphi }^{(m)}|\psi ⟩|=\prod _{\vec{k}\in \mathrm{hBZ}}{f}_{\vec{k}}\ge (1-1/\text{poly}(L){)}^{O(L^2)}=1-\frac{1}{\text{poly}(L)}.$$

Full text

1 Introduction

A generic state in quantum many-body systems is classically intractable because the dimension of the Hilbert space grows exponentially with the system size. However, physically relevant states are often non-generic in the sense of having structures, making use of which we may overcome the curse of dimensionality. Traditionally, tensor networks are used to characterize such structures and efficiently represent states in classical simulations [1]. Recently, Carleo and Troyer [2] proposed neural networks as an (alternative) ansatz for quantum many-body states. Benchmark calculations suggest that this is a promising approach.

Besides numerical experiments, it is also important to explain the working principle of neural network methods. One step in this direction is to characterize the representational power of neural networks. Here we specialize to Boltzmann machines [3], and our main contributions are

• •

We prove that any (local) tensor network state can be converted into a (local) neural network without significantly increasing the number of parameters.

• •

Despite the difficulty of representing (gapped) chiral topological states with local tensor networks [4], we construct a quasi-local neural network representation for a chiral $p$-wave superconductor.

The first result states that the representational power of neural networks is at least not weaker than that of tensor networks. The second gives a physically relevant example where neural networks may go beyond tensor networks. In combination, these results provide complementary evidence that neural networks are a promising ansatz.

2 Boltzmann machines

We provide a minimum background for those people with no prior knowledge of Boltzmann machines. The goal is to motivate the definition of neural network states, rather than a general-purpose introduction from the perspective of machine learning.

Formally, a Boltzmann machine is a type of stochastic recurrent neural network. In the language of physicists, it is a classical Ising model on a weighted undirected graph. Each vertex (also known as a unit or a neuron) of the graph carries a classical Ising variable $s_{j}=\pm 1$ and a local field $h_{j}\in\mathbb{R}$, where $j$ is the index of the vertex. For a reason that will soon be clear, the set of vertices is divided into the disjoint union of two subsets $V$ and $H$ so that $|V|+|H|$ is the total number of units. Vertices in $V$ are called visible units, and those in $H$ are called hidden units. For notational simplicity, we assume that visible units have small indices $1,2,\ldots,|V|$, and hidden units have large indices $|V|+1,|V|+2,\ldots,|V|+|H|$. Each edge of the graph carries a weight $w_{jk}\in\mathbb{R}$ that describes the interaction between $s_{j}$ and $s_{k}$. The energy of a configuration is given by

[TABLE]

A restricted Boltzmann machine [5] is a Boltzmann machine on a bipartite graph, i.e., $w_{jk}\neq 0$ only if the edge $(j,k)$ connects a visible unit and a hidden unit.

At thermal equilibrium, the configurations follow the Boltzmann distribution. Without loss of generality, we fix the temperature $T=1$. Let

[TABLE]

be the partition function. The probability of each configuration $\{s_{j}\}$ is given by $e^{-E(\{s_{j}\})}/Z$. Furthermore, the probability of each configuration $\{s_{j\leq|V|}\}$ of visible units is the marginal probability obtained by summing over hidden units:

[TABLE]

The process of training a Boltzmann machine is specified as follows. The input is a (normalized) probability distribution $Q$ over the configurations of the visible units. (Here we assume, for simplicity, that $Q$ is given. In practice, $Q$ may not be explicitly given but we are allowed to sample from $Q$.) The goal is to adjust the weights $w_{jk}$ and local fields $h_{j}$ such that the probability distribution $P$ (3) best approximates (as measured by the Kullback-Leibler divergence or some other distance function) the desired distribution $Q$. This is a variational minimization problem. We minimize a given objective function of $P$ with respect to the ansatz (3), in which the weights and local fields are variational parameters. Note that Monte Carlo techniques are usually used in training a Boltzmann machine.

An interesting question is whether an arbitrary $Q$ can be exactly represented by a Boltzmann machine. Because of the strict positivity of exponential functions, it is easy to see that the answer is no if the probability of some configuration is zero. Nevertheless,

Theorem 1** (Le Roux and Bengio [6]).**

Any probability distribution $Q$ can be arbitrarily well approximated by a restricted Boltzmann machine, provided that the number $|H|$ of hidden units is the number of configurations with nonzero probability. In general, $|H|\leq 2^{|V|}$.

This result can be slightly improved [7]. Indeed, a simple counting argument suggests that an exponential number of hidden units is necessary in general. As a function from $\{\pm 1\}^{\times|V|}$ to $[0,1]$ with one constraint (normalization), the probability distribution $Q$ has $2^{|V|}-1$ degrees of freedom. Therefore, a good approximation of $Q$ requires an exponential number of bits.

3 Neural network states

Carleo and Troyer [2] developed a minimal extension of Boltzmann machines to the quantum world. In this extension, each vertex still carries a classical spin (bit). This is very different from another extension [8], in which each vertex carries a quantum spin (qubit). It should be clear that the latter extension is more quantum. In the former extension, we use classical computers to simulate quantum many-body systems.

Expanded in the computational basis $\{\pm 1\}^{\times|V|}$, a quantum state $|\psi\rangle$ of $|V|$ qubits can be viewed as a function from $\{\pm 1\}^{\times|V|}$ to $\bar{B}(0,1)$ with one constraint (normalization), where $\bar{B}(0,1)$ denotes the closed region $|z|\leq 1$ in the complex plane. Recall that a probability distribution $Q$ is characterized by the probability of each configuration. In comparison, a state $|\psi\rangle$ is characterized by the probability amplitude of each configuration. This analog between classical probability distributions and quantum states motivates the following ansatz dubbed neural network states.

Consider a graph as before. The visible units correspond to physical qubits, and hidden units are auxiliary degrees of freedom (to be summed over). The local field $h_{j}\in\mathbb{C}$ at each vertex and the weight $w_{jk}\in\mathbb{C}$ carried by each edge are promoted to complex numbers because probability amplitudes are generally complex. An (unnormalized) neural network state based on a Boltzmann machine is given by

[TABLE]

Note that summing over hidden units is very different from taking partial trace, for the latter usually results in a mixed state. Similarly, a neural network state based on a restricted Boltzmann machine is given by Eq. (4) on a bipartite graph as described previously.

A very minor modification of the proof of Theorem 1 leads to

Corollary 1**.**

Any $N$-qubit quantum state $|\psi\rangle$ can be arbitrarily well approximated by a neural network state based on a restricted Boltzmann machine with $w_{jk}\in\mathbb{R},h_{j}\in\mathbb{C}$, provided that the number $|H|$ of hidden units is the number of configurations with nonzero probability amplitude. In general, $|H|\leq 2^{N}$.

The formalism of neural network states developed thus far applies to general quantum many-body systems. We now consider the situation that the qubits are arranged on a lattice. The lattice allows us to introduce the notion of locality [9] (with respect to the shortest path metric), which underlies almost all successful methods for simulating quantum lattice systems. Hence, it is desirable to incorporate locality into the neural network. To this end, we define a position for each unit. Let each site of the lattice carry a visible unit and some hidden units. We require that $w_{jk}\neq 0$ only if units $j$ and $k$ are close to each other. As an example, Deng et al. [10] showed that the ground state of the toric code in two dimensions can be exactly represented as a local neural network state based on a restricted Boltzmann machine.

Using variational quantum Monte Carlo techniques (“quantum Monte Carlo” is not a quantum algorithm; rather, it is just a classical Monte Carlo algorithm applied to quantum systems), Carleo and Troyer [2] performed practical calculations for quantum lattice systems in one and two spatial dimensions. For the models they studied, they observed that variational approaches based on neural networks are at least as good as those based on tensor networks. It seems worthwhile to devote more study to the practical performance of neural network states.

4 Tensor network states

We provide a very brief introduction to tensor network states following the presentation in Ref. [11], Subsection 6.3 or Ref. [12], Subsection 2.3. Then, it will be clear almost immediately that any (local) tensor network state has a (local) neural network representation.

An $l$-dimensional tensor $T$ is a multivariable function $T:\{1,2,\ldots,d_{1}\}\times\{1,2,\ldots,d_{2}\}\times\cdots\times\{1,2,\ldots,d_{l}\}\to\mathbb{C}$, and $D=\max_{j}d_{j}$ is called the bond dimension. It is easy to see that $T$ can be reshaped to an $l^{\prime}$-dimensional ($l^{\prime}=\sum_{j}\lceil\log_{2}d_{j}\rceil$ with $\lceil\cdot\rceil$ the ceiling function) tensor $T^{\prime}$ with bond dimension $2$ by representing every input variable in binary. Moreover, there is a straightforward way to identify $T^{\prime}$ with an $l^{\prime}$-qubit unnormalized state expanded in the computational basis:

[TABLE]

Thus, a neural network state with $|V|$ visible units is a $|V|$-dimensional tensor with bond dimension $2$. Indeed, each visible unit corresponds to an input variable to the tensor. As a restatement of Corollary 1,

Corollary 2**.**

Any $l$-dimensional tensor $T$ can be arbitrarily well approximated by a restricted Boltzmann machine with $w_{jk}\in\mathbb{R},h_{j}\in\mathbb{C}$, provided that the number $|H|$ of hidden units is the number of nonzero elements in $T$. In general, $|H|\leq\prod_{j=1}^{l}d_{j}$.

Informally, tensor contraction is defined as follows. Suppose we have two three-dimensional tensors $T_{1}(j_{1},j_{2},j_{3}),T_{2}(j^{\prime}_{1},j^{\prime}_{2},j^{\prime}_{3})$ of shapes $d_{1}\times d_{2}\times d_{3},d^{\prime}_{1}\times d_{2}\times d^{\prime}_{3}$, respectively. Their contraction on the middle indices is a four-dimensional tensor

[TABLE]

Similarly, suppose we have two Boltzmann machines with units $\{s_{j}\}=V\cup H,\{s^{\prime}_{j}\}=V^{\prime}\cup H^{\prime}$, weights $w_{jk},w^{\prime}_{jk}$, and local fields $h_{j},h^{\prime}_{j}$, respectively. Their contraction on the first $c$ visible units is defined as identifying $s_{j}$ with $s^{\prime}_{j}$ and then summing over $s_{j}$ for $j=1,2,\ldots,c$:

[TABLE]

where $\cdots$ is given by

[TABLE]

Note that only visible units are allowed to be contracted on, and such visible units become hidden after the contraction. Thus, the Boltzmann machine after contraction has $|V|+|V^{\prime}|-2c$ visible and $|H|+|H^{\prime}|+c$ hidden units.

A multi-qubit state $|\psi\rangle$ is a tensor network state if its tensor representation $T^{\prime}$ (5) can be obtained by contracting a network of $C$-dimensional tensors, where $C$ is a small absolute constant. The bond dimension $D$ of a tensor network is defined as the maximum bond dimension of the constituting tensors in the network.

Theorem 2**.**

Any tensor network state $|\psi\rangle$ can be arbitrarily well approximated by a neural network state based on a Boltzmann machine with $w_{jk}\in\mathbb{R},h_{j}\in\mathbb{C}$, provided that the number $|H|$ of hidden units is sufficiently large. Let $\emptyset(T_{j})$ be the number of nonzero elements in a constituting tensor $T_{j}$, and $\#(T_{j})$ be the number of elements. It suffices that

[TABLE]

where $n$ is the number of constituting tensors. Furthermore, the number of parameters in the neural network representation is upper bounded by

[TABLE]

The neural network is local (translationally invariant) if the tensor network is local (translationally invariant).

Proof.

We first represent each $T_{j}$ with a restricted Boltzmann machine as in Corollary 2, and then contract the restricted Boltzmann machines in the same way as $T_{j}$’s are contracted. Note that the Boltzmann machine after contraction is generically not restricted. The first term on the right-hand side in Eq. (9) is the total number of hidden units in all constituting restricted Boltzmann machines, and the other terms are responsible for the production of hidden units in the contraction process. Equation (10) is the total number of parameters in all constituting restricted Boltzmann machines, and the contraction process does not introduce any new parameters. It is obvious that the construction is locality and translational-invariance preserving. ∎

This result is almost optimal in the sense that the number of parameters in the neural network representation is at most a logarithmic (in the bond dimension) multiple of the number of nonzero parameters in the tensor network representation. In cases that the tensors are sparse (possibly due to the presence of symmetries), this conversion to neural network states has the desirable property of automatically compressing the representation.

As an example, we specialize Theorem 2 to matrix product states [13, 14].

Corollary 3**.**

To leading order, any $N$-qubit matrix product state with bond dimension $D$ has a neural network representation with $2ND^{2}$ hidden units and $4ND^{2}\log_{2}D$ parameters.

5 Chiral topological states

It seems difficult to obtain a local tensor network representation for gapped chiral topological states. Early attempts did not impose locality [15] or just target at expectation values of local observables rather than the wave function [16]. Recent progress [17, 4] shows that local tensor networks can describe chiral topological states, but the examples there are all gapless. Indeed, Dubail and Read [4] even proved a no-go theorem, which roughly states that for any chiral local free-fermion tensor network state, any local parent Hamiltonian is gapless. Here we construct a Boltzmann machine that approximates the unique ground state of a (gapped) chiral $p$-wave superconductor. The approximation error is inverse polynomial in the system size, and the neural network is quasi-local in the sense that the maximum distance of connections between units is logarithmic in the system size. This example explicitly demonstrates the power of Boltzmann machines.

From now on, we consider fermionic systems, for which it is necessary and there are multiple ways to account for the exchange statistics of fermions. We take a straightforward approach: Each vertex carries a Grassmann variable $\xi_{j}$ rather than an Ising variable $s_{j}$, and the sum over Ising variables in hidden units is replaced by the Grassmann integral. We work in the second quantization formalism. Let $c_{j},c_{j}^{\dagger}$ for $1\leq j\leq|V|$ be the fermionic annihilation and creation operators, and $|0\rangle$ be the vacuum state with no fermions. We identify $c_{j}^{\dagger}$ with $\xi_{j}$ so that $\Xi|0\rangle$ represents a fermionic state, where $\Xi$ is an arbitrary Grassmann variable in the algebra generated by $\xi_{1},\xi_{2},\ldots,\xi_{|V|}$. As an analog of Eq. (4),

[TABLE]

is an (unnormalized Gaussian) fermionic neural network state. Note that we have set $h_{j}=0$. This class of states appeared previously in Ref. [4].

One of the simplest examples of a chiral topological phase is the $p+ip$ superconductor [18]. For concreteness, we consider the lattice model in Ref. [19]. Let $c_{\vec{x}},c_{\vec{x}}^{\dagger}$ be the fermionic annihilation and creation operators at site $\vec{x}\in\mathbb{Z}^{2}$ on a two-dimensional square lattice. Let $\vec{i}=(1,0)$ and $\vec{j}=(0,1)$ be the unit vectors in the $x$ and $y$ axes, respectively. The Hamiltonian is

[TABLE]

Let $\vec{k}=(k_{x},k_{y})$ be the lattice momentum, and $\int_{\textnormal{BZ}}\mathrm{d}\vec{k}$ be the integral over the Brillouin zone $(-\pi,\pi]^{\times 2}$. To slightly simplify the presentation, we will be sloppy about unimportant overall prefactors in the calculations below. The Fourier transform

[TABLE]

leads to

[TABLE]

in the momentum space, where

[TABLE]

The quasiparticle spectrum is given by

[TABLE]

Hence, $H$ is gapless for $\mu=0,\pm 2$ and gapped otherwise. The unique ground state of $H$ is

[TABLE]

where $|0\rangle$ is the vacuum state, and

[TABLE]

so that $u_{\vec{k}}=u_{-\vec{k}}$ and $v_{\vec{k}}=-v_{-\vec{k}}$. It is not difficult to see that the model (12) is a trivial superconductor for $|\mu|>2$. It is topological superconductors with opposite chirality for $-2<\mu<0$ and $0<\mu<2$, respectively.

Indeed, a state of the form (17) has an exact local neural network representation (11) [4] if $u_{\vec{k}},v_{\vec{k}}$ are constant-degree trigonometric polynomials, i.e., polynomials in $e^{\pm ik_{x}},e^{\pm ik_{y}}$. In particular, we have each site $\vec{x}$ carry a visible and a hidden unit. To simplify the notation, the Grassmann variable in the visible unit is identified with $c_{\vec{x}}^{\dagger}$, and that in the hidden unit is denoted by $\xi_{\vec{x}}$. The Fourier transform of Grassmann variables is defined as

[TABLE]

In the momentum space, it is easy to see that the state (17) can be represented as

[TABLE]

where $\int[\mathrm{d}\xi_{\vec{k}}]$ denotes the integral over all Grassmann variables in the momentum space with a proper measure. Transforming to the real space, $[\mathrm{d}\xi_{\vec{k}}]$ becomes $\prod_{\vec{x}\in\mathbb{Z}^{2}}\mathrm{d}\xi_{\vec{x}}$, and the exponent in parentheses is local because $u_{\vec{k}},v_{\vec{k}}$ are trigonometric polynomials (the maximum distance of connections between units is proportional to the degree of the trigonometric polynomials). Thus, Eq. (20) reduces to Eq. (11) with additional properties: (i) the neural network representation is translationally invariant; (ii) there are no connections between visible units.

In the remainder of this section, standard asymptotic notations are used extensively. Let $f,g:\mathbb{R}^{+}\to\mathbb{R}^{+}$ be two functions. One writes $f(x)=O(g(x))$ if and only if there exist constants $M,x_{0}>0$ such that $f(x)\leq Mg(x)$ for all $x>x_{0}$; $f(x)=\Omega(g(x))$ if and only if there exist constants $M,x_{0}>0$ such that $f(x)\geq Mg(x)$ for all $x>x_{0}$; $f(x)=o(g(x))$ if and only if for any constant $M>0$ there exists a constant $x_{0}>0$ such that $f(x)<Mg(x)$ for all $x>x_{0}$.

Generically, exact representations are too much to ask for; hence, approximate representations are acceptable. Furthermore, we usually have to work with a finite system size in order to rigorously quantify the approximation error and the succinctness of the representation. We now justify these statements with an example. Matrix product states are an excellent ansatz for ground states in one-dimensional gapped systems [20]. A generic gapped ground state in a chain of $N$ spins (i) cannot be exactly written as a matrix product state with bond dimension $e^{o(N)}$; (ii) is not expected to be approximated (in the sense of $99\%$ fidelity) by a matrix product state with bond dimension $O(1)$ because errors may accumulate while we truncate the Schmidt coefficients across every cut [21]; (iii) can be approximated by a matrix product state with bond dimension $N^{o(1)}$ [22, 23].

The goal is to construct an as-local-as-possible neural network representation (11) for the ground state of the model (12).

Proposition 1**.**

Let $|\psi\rangle$ be the ground state of the model (12) on a (finite) square lattice of size $L\times L$ with periodic boundary conditions. There exists a neural network state $|\phi\rangle$ such that the fidelity $|\langle\phi|\psi\rangle|\geq 1-1/\poly(L)$ and that the maximum distance of connections between units is $O(\epsilon^{-1}\log L)$, where $\epsilon$ is the energy gap.

Proof.

For concreteness, we consider $0<\mu<2$. It should be clear that the same result holds for other values of $\mu$ provided that the system is gapped. The main idea is to approximate $u_{\vec{k}}$ (18) with a trigonometric polynomial. The approximation error is exponentially small in the degree of the polynomial because $u_{\vec{k}}$ is a real analytic function of $\vec{k}$. The locality of the neural network is due to the smallness of the degree.

We now provide the details of the construction. Assume $L$ is odd. This slightly simplifies the analysis, but it should be clear that a minor modification of the analysis leads to the same result for even $L$. We abuse the notation by letting BZ denote the set of viable points $\{0,\pm 2\pi/L,\ldots,\pm(L-1)\pi/L\}^{\times 2}$ for lattice momenta in the Brillouin zone, and

[TABLE]

be the “right half” of BZ. The Fourier transform (13) becomes discrete:

[TABLE]

and the Hamiltonian in the momentum space is given by

[TABLE]

The normalized ground state is

[TABLE]

This equation should be understood as the $\vec{k}=\vec{0}$ mode of $|\psi\rangle$ being vacant. Consider the normalized state

[TABLE]

where $u^{(m)}_{\vec{k}}$ is a degree-$m$ trigonometric polynomial obtained by expanding $u_{\vec{k}}$ in Fourier series and computing the partial sum up to order $m$. Similar to Eq. (20), $|\phi^{(m)}\rangle$ has an exact neural network state representation (11) such that the maximum distance of connections between units is $O(m)$.

As $u_{\vec{k}}$ is a real analytic function of $\vec{k}$, its Fourier coefficients of order $m$ decay exponentially as $e^{-m/\xi}$ for some constant $\xi>0$. The decay rate $\xi$ is a function of $\mu$ and can be solved analytically; see, e.g., Ref. [24], Chapter 2. In the regime the energy gap $\epsilon$ is small, we obtain $\xi=O(1/\epsilon)$. Therefore,

[TABLE]

Furthermore, the absolute values of $u_{\vec{k}},v_{\vec{k}}$ are bounded away from [math]:

[TABLE]

Equations (26), (27) imply

[TABLE]

for $m=O(\epsilon^{-1}\log L)$ with a sufficiently large constant prefactor hidden in the big-O notation. As $|\psi\rangle,|\phi^{(m)}\rangle$ are product states in the momentum space, the fidelity is given by

[TABLE]

We complete the proof by letting $|\phi\rangle=|\phi^{(m)}\rangle$. ∎

It is not difficult to see that $|\psi\rangle$ is well approximated by a thermal state at inverse temperature $O(\log L)$. Since the thermal state has a projected entangled pair approximation with quasi-polynomial bond dimension $e^{O(\log^{2}L)}$ [25, 26, 27], there exists a projected entangled pair state $|\varphi\rangle$ with bond dimension $e^{O(\log^{2}L)}$ such that $|\langle\varphi|\psi\rangle|\geq 1-1/\poly(L)$. It is an open problem to improve the bond dimension of $|\varphi\rangle$ to $\poly(L)$.

Acknowledgments and notes

The authors would like to thank Xie Chen for collaboration in the early stages of this project. Y.H. acknowledges funding provided by the Institute for Quantum Information and Matter, an NSF Physics Frontiers Center (NSF Grant PHY-1733907) with support of the Gordon and Betty Moore Foundation (GBMF-2644). J.E.M. is supported by NSF DMR-1507141, DMR-1918065 and a Simons Investigatorship.

Part of this work was presented on November 27, 2014 (Thanksgiving day!) at the Perimeter Institute for Theoretical Physics. Very recently, we became aware of some related papers [28, 29, 30], which studied the relationship between neural and tensor network states using different methods. In particular, Theorem 2 and Corollary 3 are stronger than Theorem 3 in Ref. [30]. After the present work had been on arXiv, some other related papers [31, 32, 33, 34, 35, 36, 37, 38] appeared.