Near-Term Quantum-Classical Associative Adversarial Networks

TL;DR

This paper presents a hybrid quantum-classical adversarial network architecture that leverages a small quantum Boltzmann machine to improve learning quality on image datasets, suitable for near-term quantum devices.

Contribution

Introduction of QAAN, a novel hybrid quantum-classical adversarial network architecture with a small quantum Boltzmann machine integrated into the discriminator.

Findings

QAAN outperforms classical GANs on MNIST and CIFAR-10 datasets.

QAAN achieves higher Inception scores and lower Fréchet Inception distances.

Model is feasible for implementation on current quantum hardware.

Abstract

We introduce a new hybrid quantum-classical adversarial machine learning architecture called a quantum-classical associative adversarial network (QAAN). This architecture consists of a classical generative adversarial network with a small auxiliary quantum Boltzmann machine that is simultaneously trained on an intermediate layer of the discriminator of the generative network. We numerically study the performance of QAANs compared to their classical counterparts on the MNIST and CIFAR-10 data sets, and show that QAANs attain a higher quality of learning when evaluated using the Inception score and the Fr\'{e}chet Inception distance. As the QAAN architecture only relies on sampling simple local observables of a small quantum Boltzmann machine, this model is particularly amenable for implementation on the current and next generations of quantum devices.

Equations

$$E\left(\mathbf{z};\mathbf{\theta }\right)=-\sum _a{b}_a{z}_a-\sum _{a<b}{W}_{ab}{z}_a{z}_b,$$

$$E\left(\mathbf{v},\mathbf{h};\mathbf{\theta }\right)=-\sum _{\upsilon }{b}_{\upsilon }{v}_{\upsilon }-\sum _{\eta }{b}_{\eta }{h}_{\eta }-\sum _{\upsilon ,\eta }{W}_{\upsilon \eta }{v}_{\upsilon }{h}_{\eta }.$$

$$P\left(\mathbf{v};\mathbf{\theta }\right)=\frac{1}{Z}\sum _{\mathbf{h}}{\text{e}}^{-E\left(\mathbf{v},\mathbf{h};\mathbf{\theta }\right)},$$

$$\mathcal{L}\left(\mathbf{\theta }\right)=-\sum _{\mathbf{v}}{P}_{\text{data}}\left(\mathbf{v}\right)\log \left(P\left(\mathbf{v};\mathbf{\theta }\right)\right).$$

$${\partial }_{\theta }\mathcal{L}\left(\mathbf{\theta }\right)=\sum _{\mathbf{v}}{P}_{\text{data}}\left(\mathbf{v}\right)\text{expectationvalue}{{\partial }_{\theta }E\left(\mathbf{v},\mathbf{h};\mathbf{\theta }\right)}_{\mathbf{v}}-\text{expectationvalue}{\partial }_{\theta }E\left(\mathbf{v},\mathbf{h};\mathbf{\theta }\right).$$

$$\begin{array}{rl}{J}^{\left(\mathcal{G}\right)}& ({\mathbf{\theta }}^{\left(\mathcal{G}\right)},{\mathbf{\theta }}^{\left(\mathcal{D}\right)})=-J^{\left(\mathcal{D}\right)}({\mathbf{\theta }}^{\left(\mathcal{G}\right)},{\mathbf{\theta }}^{\left(\mathcal{D}\right)})\\ & =\text{expectationvalue}{\log \left(\mathcal{D}\left(\mathbf{x}\right)\right)}_{\text{data}}+\text{expectationvalue}{\log \left(1-\mathcal{D}\left(\mathcal{G}\left(\mathbf{z}\right)\right)\right)}_{\text{noise}},\end{array}$$

$${\mathbf{\theta }}^{\left(\mathcal{G}\right)\ast }=\underset{{\mathbf{\theta }}^{\left(\mathcal{G}\right)}}{\mathrm{argmin}}\underset{{\mathbf{\theta }}^{\left(\mathcal{D}\right)}}{\max }{J}^{\left(\mathcal{G}\right)}({\mathbf{\theta }}^{\left(\mathcal{G}\right)},{\mathbf{\theta }}^{\left(\mathcal{D}\right)}).$$

$$\mathcal{D}=\mathcal{C}\circ \mathcal{F}.$$

$$H\left(\mathbf{\theta }\right)=-\sum _a{\Gamma }_a{\sigma }_a^x-\sum _a{b}_a{\sigma }_a^z-\sum _{a<b}{W}_{ab}{\sigma }_a^z{\sigma }_b^z$$

$$\rho \left(\mathbf{\theta }\right)=\frac{{\text{e}}^{-H\left(\mathbf{\theta }\right)}}{\text{tr}\left({\text{e}}^{-H\left(\mathbf{\theta }\right)}\right)},$$

$$P\left(\mathbf{v};\mathbf{\theta }\right)=\text{tr}\left({\Pi }_{\mathbf{v}}\rho \left(\mathbf{\theta }\right)\right)$$

$$\mathcal{L}\left(\mathbf{\theta }\right)\le \tilde{\mathcal{L}}\left(\mathbf{\theta }\right)\equiv -\sum _{\mathbf{v}}{P}_{\text{data}}\left(\mathbf{v}\right)\left(\frac{\text{tr}\left({\text{e}}^{-H_{\mathbf{v}}}\left(\mathbf{\theta }\right)\right)}{\text{tr}\left({\text{e}}^{-H\left(\mathbf{\theta }\right)}\right)}\right),$$

$$H_{\mathbf{v}}\left(\mathbf{\theta }\right)=H\left(\mathbf{\theta }\right)-\ln \left({\Pi }_{\mathbf{v}}\right)=H\left({\sigma }_{\upsilon }^x\to 0,{\sigma }_{\upsilon }^z\to v_{\upsilon };\mathbf{\theta }\right)$$

$${\partial }_{\theta }\tilde{\mathcal{L}}\left(\mathbf{\theta }\right)=\sum _{\mathbf{v}}{P}_{\text{data}}\left(\mathbf{v}\right)\text{expectationvalue}{{\partial }_{\theta }{H}_{\mathbf{v}}\left(\mathbf{\theta }\right)}_{\mathbf{v}}-\text{expectationvalue}{\partial }_{\theta }H\left(\mathbf{\theta }\right),$$

$$P_{\text{data}}\left(\mathbf{z};\left\{{\mathbf{s}}^i\right\}\right)=\sum _{i=1}^N{q}^{d\left(\mathbf{z},{\mathbf{s}}^i\right)}{\left(1-q\right)}^{\dim \left(\mathbf{z}\right)-d\left(\mathbf{z},{\mathbf{s}}^i\right)}$$

$${\partial }_{b_{\upsilon }}\mathcal{L}\left(\mathbf{\theta }\right)$$

$${\partial }_{b_{\eta }}\mathcal{L}\left(\mathbf{\theta }\right)$$

$${\partial }_{W_{\upsilon \eta }}\mathcal{L}\left(\mathbf{\theta }\right)$$

$$P\left(h_{\eta }=1|\mathbf{v}\right)$$

$$P\left(v_{\upsilon }=1|\mathbf{h}\right)$$

$$-{\partial }_{b_a}\tilde{\mathcal{L}}\left(\mathbf{\theta }\right)$$

$$-{\partial }_{W_{ab}}\tilde{\mathcal{L}}\left(\mathbf{\theta }\right)$$

$$\text{expectationvalue}{{\sigma }_{\eta }^z}_{\mathbf{v}}=\frac{b_{\eta }^{\text{eff}}}{D_{\eta }}\tanh \left(D_{\eta }\right),$$

$$b_{\eta }^{\text{eff}}$$

$$D_{\eta }$$

$$p_{\beta }\left(\left\{{\mathbf{z}}^m\right\}\right)=\frac{{\text{e}}^{-\beta \left(E_{\text{cl}}\left(\left\{{\mathbf{z}}^m\right\}\right)+E_{\text{qm}}\left(\left\{{\mathbf{z}}^m\right\};\beta \right)\right)}}{\sum _{\left\{{\mathbf{z}}^m\right\}}{\text{e}}^{-\left(E_{\text{cl}}\left(\left\{{\mathbf{z}}^m\right\}\right)+E_{\text{qm}}\left(\left\{{\mathbf{z}}^m\right\};\beta \right)\right)}},$$

$$E_{\text{cl}}\left(\left\{{\mathbf{z}}^m\right\};\mathbf{\theta }\right)$$

$$E_{\text{qm}}\left(\left\{{\mathbf{z}}^m\right\};\mathbf{\theta };\beta \right)$$

$$\begin{array}{rl}\mathrm{IS}\left(\mathcal{G}\right)& =\exp \left(\text{expectationvalue}{D_{\mathrm{KL}}\left(p\left(y|\mathbf{x}\right)||p\left(y\right)\right)}_{\mathbf{x}}\right)\\ & =\exp \left(S\left(y\right)-\text{expectationvalue}{S\left(y|\mathbf{x}\right)}_{\mathbf{x}}\right),\end{array}$$

$$\begin{array}{rl}\mathrm{FID}\left(\mathcal{G}\right)& =\text{norm}{{\mathbf{\mu }}_{\text{data}}-{\mathbf{\mu }}_{\mathcal{G}}}^2\\ & +\text{Tr}\left({\Sigma }_{\text{data}}+{\Sigma }_{\mathcal{G}}-2{\left({\Sigma }_{\text{data}}{\Sigma }_{\mathcal{G}}\right)}^{1/2}\right).\end{array}$$

Full text

††preprint: MIT-CTP/5125

I Introduction

Quantum mechanics is widely believed to produce distributions of data difficult to replicate classically Boixo et al. (2018). As deep learning is implemented through applying a series of transformations to latent probability distributions to approximate empirical distributions given by data, this has led to a trend in recent years of applying quantum computing techniques to machine learning Biamonte et al. (2017); Wiebe et al. (2016); Farhi and Neven (2018); Pudenz and Lidar (2013); Neven et al. (2012); Lloyd et al. (2013); Denchev et al. (2012); Killoran et al. (2018); Rebentrost et al. (2018). One approach to this task consists of applying well-known quantum algorithms—such as the HHL algorithm Harrow et al. (2009)—to replace their classical counterparts—such as the BLAS library Biamonte et al. (2017). This method has been applied to principal component analysis Lloyd et al. (2014), support vector machines Rebentrost et al. (2014), and most recently, generative adversarial networks Dallaire-Demers and Killoran (2018); Lloyd and Weedbrook (2018). These architectures, however, require large quantum networks and the extensive use of quantum memory, which are not yet available on the current generation of quantum devices Aaronson (2015).

Instead, one could focus on translating such classical machine learning algorithms for use on current and near-term quantum devices, in the so-called Noisy Intermediate-Scale Quantum (NISQ) era Preskill (2018). This has been made possible by recent developments in neutral atom and ion trap architectures Endres et al. (2016); Zhang et al. (2017), as well as gate-model processors Neill et al. (2018); Kandala et al. (2017). Inspired by variational hybrid quantum-classical algorithms Farhi et al. (2014); McClean et al. (2016); Peruzzo et al. (2014), there has been an effort to create machine learning models that use small quantum devices to aid in the training of classical machine learning architectures. Recently studied hybrid architectures include generalizations of Helmholtz machines Benedetti et al. (2018); Perdomo-Ortiz et al. (2018) and variational autoencoders Khoshaman et al. (2018). In these models, the small quantum device encodes a quantum Boltzmann machine (QBM) Amin et al. (2018), which is a quantum generalization of classical restricted Boltzmann machines (RBMs) Smolensky (1986). In this paper, we focus on using small QBMs to improve the performance of generative adversarial networks (GANs).

GANs are the state-of-the-art classical machine learning architectures for unsupervised learning Arjovsky et al. (2017); Goodfellow et al. (2016); Goodfellow (2016); Gulrajani et al. (2017); Kodali et al. (2018); Metz et al. (2016); Goodfellow et al. (2014). They have a wide range of applications, including regression and classification Radford et al. (2015); Salimans et al. (2016), image generation Denton et al. (2015); Reed et al. (2016); Karras et al. (2018), and image-to-image translation Isola et al. (2017); Li and Wand (2016). GANs can be thought of as a zero-sum game between two players—the generator and the discriminator—each implemented as a neural network. Taking the concrete setting of image generation, the generator learns to create images resembling a given data set of authentic images, and the discriminator learns to distinguish between images produced by the generator and images drawn from the true data set Goodfellow et al. (2014). The generator does not have direct access to the data set, and only learns how to create images through feedback from the discriminator—that is, through an error signal backpropagated through the GAN.

Although GANs are the most ubiquitous adversarial models, they are notoriously difficult to train Salimans et al. (2016). One of the major reasons GANs are difficult to train is that the generator and discriminator learn at different rates generically. One approach towards combating this issue is through the use of associative adversarial networks (AANs) Arici and Celikyilmaz (2016). In this architecture, a Boltzmann machine acts as an associative memory that learns the high-level feature distribution of a layer of the discriminator. The generator then draws samples from the distribution approximated by the Boltzmann machine. The associative memory also adds expresivity to the network by providing the generator with inputs drawn from a more meaningful, data-specific distribution, rather than a uniform or Gaussian distribution as is the case for standard GANs. Motivated by the observed improvement in performance of QBMs over RBMs Amin et al. (2018); Kieferová and Wiebe (2017), we propose a method of implementing hybrid quantum-classical AANs (QAANs), where the associative memory is instead provided by a QBM.

The structure of the paper is as follows. In Sec. II, we give an introduction to classical adversarial networks and, in particular, AANs. In Sec. III, we construct a quantum analog of AANs, and give a numerical comparison between classical and quantum AANs in Sec. IV. Finally, we discuss our results and future research directions in Sec. V.

II Classical Generative Models

II.1 Restricted Boltzmann Machines

A Boltzmann machine is an energy-based generative model, and one of the first neural networks capable of learning internal representations for and sampling from arbitrary probability distributions Hinton and Sejnowski (1986). Recent work has seen successful applications of this model to a wide variety of machine learning tasks, including image Hinton et al. (2006), text Salakhutdinov and Hinton (2009), and speech Mohamed et al. (2012) generation. It also serves as a key component in other machine learning architectures, such as deep belief networks Hinton et al. (2006). A Boltzmann machine is characterized by an energy function

[TABLE]

where $\bm{z}\in\left\{0,1\right\}^{N}$ is a binary vector and $\bm{\theta}=\left\{\bm{b},\bm{W}\right\}$ are the model parameters. The model can be viewed as a two-layer network by dividing its nodes into visible units, which represent the input data, and hidden units, which form an internal representation of the data. In practice, since training a general Boltzmann machine is impractical, we consider restricted Boltzmann machines (RBMs) which further simplify the model to only contain connections between visible and hidden units Smolensky (1986) (see Fig. 1). By labeling the indices of visible nodes by $\upsilon$ and the indices of hidden nodes by $\eta$, we can separate our vector as $\bm{z}=\left(\bm{v};\bm{h}\right)$, and rewrite the energy function in the form:

[TABLE]

For an input vector $\bm{v}$, this network assigns the probability

[TABLE]

where $Z=\sum\limits_{\bm{v}}\sum\limits_{\bm{h}}\textrm{e}^{-E\left(\bm{v},\bm{h};\bm{\theta}\right)}$ is the partition function. The model parameters $\bm{\theta}$ are then chosen such that samples drawn from the marginal probability distribution $P\left({\bm{v}};\bm{\theta}\right)$ approximate samples drawn from the empirical probability distribution of the data $P_{\textrm{data}}\left(\bm{v}\right)$. This is achieved by minimizing the Kullback–Leibler divergence (KL) divergence Kullback and Leibler (1951) between $P_{\textrm{data}}\left(\bm{v}\right)$ and $P\left({\bm{v}};\bm{\theta}\right)$, or equivalently, by minimizing the negative log-likelihood

[TABLE]

The minimization of $\mathcal{L}\left(\bm{\theta}\right)$ is usually performed using a gradient based optimizer, where the gradient of the loss function is:

[TABLE]

Here, $\expectationvalue{\cdot}$ denotes the average with respect to a Boltzmann distribution with the energy given by Eq. (2), and $\expectationvalue{\cdot}_{\bm{v}}$ is the same but with the visible nodes clamped to $\bm{v}$. The first term is known as the positive phase, and the second term as the negative phase. Exact maximum likelihood learning of the negative phase requires knowledge of the partition function, which is intractable. Therefore, in practice, this phase is approximated using Gibbs sampling Geman and Geman (1984). We use Persistent Contrastive Divergence (PCD) Tieleman (2008a) to train our RBM and simulated annealing to sample from it; both are described in Appendix A.

II.2 Generative Adversarial Networks

Generative adversarial networks (GANs) are structured probabilistic models over the space of observed variables $\bm{x}$ and latent variables $\bm{z}$ Goodfellow et al. (2014); Goodfellow (2016). They implicitly model high-dimensional distributions of data and can be used to efficiently generate samples from these distributions. GANs are generally characterized by a pair of neural networks competing against each other in a zero-sum game. The generator network $\mathcal{G}$ with parameters $\bm{\theta}^{\left(\mathcal{G}\right)}$ learns to map vectors $\bm{z}$ from the latent space to samples $\bm{x}=\mathcal{G}\left(\bm{z};\bm{\theta}^{\left(\mathcal{G}\right)}\right)$ drawn from a probability distribution that is close to $P_{\textrm{data}}$. The discriminator network $\mathcal{D}$ with parameters $\bm{\theta}^{\left(\mathcal{D}\right)}$ receives samples from both the generator and the true distribution of data, and emits a probability $\mathcal{D}\left(\bm{x};\bm{\theta}^{\left(\mathcal{G}\right)}\right)\in\left[0,1\right]$ that the input is real. The generator wishes to minimize its loss function $J^{\left(\mathcal{G}\right)}\left(\bm{\theta}^{\left(\mathcal{G}\right)},\bm{\theta}^{\left(\mathcal{D}\right)}\right)$ by attempting to fool the discriminator into believing that its generated samples are real. The discriminator, on the other hand, tries to minimize its loss function $J^{\left(\mathcal{D}\right)}\left(\bm{\theta}^{\left(\mathcal{G}\right)},\bm{\theta}^{\left(\mathcal{D}\right)}\right)$ by correctly classifying the inputs as either fake or real Goodfellow et al. (2016). We can cast these statements into the minimax optimization problem Goodfellow (2016):

[TABLE]

where the goal is to find the optimal generator parameters

[TABLE]

Here, $\expectationvalue{\cdot}_{\textrm{data}}$ denotes the expectation value when $\bm{x}$ is drawn from real data, and $\expectationvalue{\cdot}_{\textrm{noise}}$ denotes the expectation value when $\bm{z}$ is drawn from the distribution of latent variables. For GANs, the latent variables are normally chosen to be Gaussian or uniform noise Goodfellow et al. (2014). The first term in Eq. (6) favors the discriminator outputting $1$ on real data, while the second term favors the discriminator outputting [math] on generated data. The generator strives to achieve the opposite. The solution to this optimization problem is a point of Nash equilibrium where the generator samples are indistinguishable from the real data and the discriminator predicts $0.5$ on all inputs.

Unfortunately, finding the Nash equilibrium is quite difficult in practice Salimans et al. (2016). In recent years, there have been many proposals aiming to improve the training of GANs. Some of them include considering a non-saturating version of the generator loss function Goodfellow (2016), introducing surrogate objective functions Metz et al. (2016), regularizing the discriminator by adding gradient penalty terms Gulrajani et al. (2017); Kodali et al. (2018), and using a formulation of the training objective based on the Wasserstein-1 distance Arjovsky et al. (2017); Gulrajani et al. (2017). Most of these architectures are extensions of deep convolutional GANs (DCGANs) Radford et al. (2015), which use convolutional neural networks (CNNs) for their generators and discriminators, instead of the fully-connected dense layers proposed initially Goodfellow et al. (2014). Our implementation of a DCGAN is described in Fig. 3. In this paper we focus on the AAN architecture Arici and Celikyilmaz (2016), which is an extension of the DCGAN architecture, and the improvements that it brings to training GANs, as described in Section II.3.

II.3 Associative Adversarial Networks

An associative memory provided by an RBM can circumvent the imbalance in training rates typically present in a GAN and enhance the expressivity of the model; such architectures are called associative adversarial networks (AANs) Arici and Celikyilmaz (2016). In an AAN, the latent space for the generator is treated as a feature space that is learned by the RBM. The RBM is simultaneously trained with the GAN on an intermediate layer of the discriminator, with a sigmoid activation function interpreted as the probability of a particular neuron firing. In general, in an AAN, the discriminator $\mathcal{D}$ is decomposed into a mapping into the feature space $\mathcal{F}$ and a a classifier $\mathcal{C}$ such that:

[TABLE]

We find that in practice, however, using a trivial classifier as in Fig. 3b suffices to improve the performance of the generator.

Though the initial motivation behind AANs was to bring the generator and discriminator learning rates in line with each other—as often the instability in GAN training is due to the discriminator learning more quickly than the generator—our DCGAN implementation for the data sets under consideration experiences no notable difference in the learning rates of the generator and discriminator, and thus our main advantage when using an AAN is due to the generator expressivity gained when it draws samples from an improved feature space rather than from noise. Our AAN architecture consists of the DCGAN architecture described in Fig. 3, coupled to an RBM as in Fig. 4.

III Quantum Generative Models

III.1 Quantum Boltzmann Machines

Quantum Boltzmann machines (QBMs) are a recently introduced method of quantizing Boltzmann machines that have been numerically observed to give a quantum speedup in both the rate of training and in the accuracy of the approximating distributions Amin et al. (2018); Kieferová and Wiebe (2017). As initially proposed, instead of considering the classical energy function of Eq. (1), one considers the Hamiltonian:

[TABLE]

and the thermal density matrix:

[TABLE]

where now $\bm{\theta}=\left\{\bm{\varGamma},\bm{b},\bm{W}\right\}$ are the model parameters. Defining the projector onto the subspace with visible nodes equal to $\bm{v}$ as $\varPi_{\bm{v}}$, our goal is to now train the parameters $\bm{\varGamma}$, $\bm{b}$, and $\bm{W}$ such that the probability distribution of samples from $\rho$

[TABLE]

approximates the empirical probability distribution $P_{\textrm{data}}\left(\bm{v}\right)$. Due to the difficulties in computing the log-liklihood of this distribution Amin et al. (2018), one instead usually 111The recently introduced relative entropy training Kieferová and Wiebe (2017); Wiebe and Wossnig (2019) is approximation free, but we do not study its behavior in this work. trains QBMs to minimize the upper bound on the loss function

[TABLE]

where

[TABLE]

is the clamped Hamiltonian. The gradients of this loss function are now given by:

[TABLE]

where here $\expectationvalue{\cdot}$ denotes expectation values taken with respect to $\rho$ and $\expectationvalue{\cdot}_{\bm{v}}$ denotes expectation values taken with respect to the thermal density matrix with clamped Hamiltonian $H_{\bm{v}}$. Training on this upper bound leads to $\bm{\varGamma}\to\bm{0}$, so in training $\bm{\varGamma}$ is fixed to some constant and treated as a learning hyperparameter Amin et al. (2018).

In order to numerically simulate large QBMs on a classical computer, we consider only the stoquastic Hamiltonian given by Eq. (9), and not more general QMA-hard Hamiltonians which have been studied in the context of QBMs Kieferová and Wiebe (2017); Wiebe et al. (2019). The details of our Monte Carlo-based simulation method are given in Appendix B.

III.2 Quantum-Classical Associative Adversarial Networks

We now quantize AANs by transforming the associated RBM into a QBM, and call the resulting architecture a quantum-classical associative adversarial network (QAAN). Our implementation otherwise exactly follows that of our AAN (see Sec. II.3).

The hybrid quantum-classical nature of our QAAN architecture lends itself well to being implementable on NISQ devices. Though in general simulating quantum thermal states is inefficient even on quantum devices, there is evidence that the structure of QBMs allows for efficient heuristic training on NISQ devices Anschuetz and Cao (2019). Furthermore, as QBM training only necessitates measuring simple local observables in the QBM state, there is no need for a quantum memory for the training data. Finally, as the QBM is only trained on a feature space of much lower dimensionality than the data, many fewer qubits are required to implement the QBM than if it were to directly learn the data. This hybrid quantum-classical approach to machine learning, inspired by variational hybrid quantum-classical algorithms Farhi et al. (2014); McClean et al. (2016); Peruzzo et al. (2014), can serve as a model for similar future quantum machine learning architectures that minimize the need for large quantum devices.

IV Results

We compare the performance of the classical and quantum-classical architectures on three data sets of increasing difficulty. First, we show that QBMs outperform RBMs on a simple synthetic data set of mixed Bernoulli distributions, thus suggesting that quantum models can provide an improvement in approximating certain distributions. Next, we compare the performance of DCGAN, AAN, and QAAN architectures on the MNIST data set LeCun (1998), which is a standard benchmark used in classical machine learning. Finally, we test the three architectures on the more challenging CIFAR-10 data set Krizhevsky and Hinton (2009), and show that our implementation of a QAAN architecture produces samples that more closely mirror the samples drawn from the data distributions.

In order to quantitatively evaluate the different architectures on real data sets, we use the Inception score Salimans et al. (2016) and the Fréchet Inception distance (FID) Heusel et al. (2017). The Inception score computes the KL divergence between the conditional class distribution and the marginal class distribution over generated samples as predicted by an Inception-v3 network Szegedy et al. (2016). A higher score indicates better generated images. This is the most widely used metric for evaluating GANs and allows for easy comparisons with previous works. However, the IS has some limitations in assessing the realism and intra-class diversity of the generated samples. The FID is a more comprehensive metric that has been shown to mitigate some of these shortcomings Heusel et al. (2017). It relies on computing the Wasserstein-2 distance between the generated data and the real data in the feature space of an Inception-v3 network. Lower FID values are better and suggest that the generated images are more similar to the original data set. A detailed description of both metrics is provided in Appendix D.

In all of our experiments, $10^{4}$ samples are randomly drawn from each model and used to evaluate its performance. Similar results are obtained for different initializations of the network parameters. The Inception score showed a higher variance over different data subsets and therefore we report the average over $10$ batches, each consisting of $1000$ generated images. Since the FID relies on a particular layer of a pre-trained network and can only be unambigously defined for colored images, we only use it in quantifying the performace of our models on CIFAR-10.

IV.1 Synthetic Data

We begin by comparing the learning capability of an RBM with that of a QBM by training both on $6400$ samples from the mixed Bernoulli distribution:

[TABLE]

with $N$ random modes $\left\{\bm{s}^{i}\right\}$, where $d\left(\bm{a},\bm{b}\right)$ denotes the Hamming distance between $\bm{a}$ and $\bm{b}$. We choose $N=8$ and $q=0.9$ for our numerical experiments. More details about our training procedure are provided in Appendix C. Samples are then drawn from each of the data, RBM, and QBM distributions. The resulting empirical probability distributions are given by Fig. 5. To quantitatively measure the distance between the original and reconstructed distributions, we compute the KL divergence $D_{\mathrm{KL}}\left(\left.\left.P_{\textrm{data}}\right|\right|P_{\textrm{BM}}\right)$. We obtain a KL divergence of approximately $1.23$ for the RBM and $0.76$ for the QBM. We see that the QBM outperforms the RBM, even though both of them have the same number of parameters and are trained similarly.

IV.2 MNIST Data

We now compare our implementations of DCGAN, AAN, and QAAN. We train all three networks on the MNIST handwritten digit data set LeCun (1998), which consists of $6\times 10^{4}$ grayscale images, each of size $28\times 28$ pixels. We rescale the pixel values to the interval $\left[-1,1\right]$ before feeding them into our networks. The training parameters for this data set are summarized in Appendix C.

We monitor the performance of each network by computing the Inception score on generated images after every epoch, as shown in Fig. 6. We notice that the results converge within the considered $30$ epochs, with QAAN reaching a better Inception score than its classical counterpart by roughly $2\%$. It is worth mentioning that all of the architectures considered achieve a score that is close to that of real data ($9.75$). We also visually examine samples of generated handwritten digits, as shown in Fig. 7, and confirm that they are almost indistinguishable from the orginal data.

IV.3 CIFAR-10 Data

Finally, we study the performance of our models on the CIFAR-10 data set. This data set consists of $6\times 10^{4}$ colored natural scene images, each of size $32\times 32$ pixels and $3$ color channels, divided across $10$ different classes Krizhevsky and Hinton (2009). The training procedure is identical to the case of MNIST and is described in Appendix C.

We once again compute the Inception score and FID on generated images after every epoch, and plot our results for all three models in Fig. 8. Notice that the inception scores are overall lower than in the case of MNIST, even though both data sets have $10$ classes. This can be attributed to the increased complexity of the data set of colored images. The better performance of QAAN is more prominent in the FID metric, where it achieves a consistently better score in the last five training epochs when compared to DCGAN and AAN. The improvement in FID is approximately $7\%$ by the end of training. The metrics reported in Fig. 8 are on par with those obtained for a variety of classical GANs, such as WGANs, with similar architectures Fedus et al. (2017); Ostrovski et al. (2018). We also note that both AAN and QAAN have a steeper learning curve in the early epochs, which can be attributed to the associative memory providing a more meaningful latent space for the generator. Sample images from CIFAR-10 and our models are shown in Fig. 9. The generated images look very realistic.

V Discussion

In Sec. IV we showed that the QAAN architecture can learn the MNIST and CIFAR-10 data sets more effectively than the AAN and DCGAN architectures. Since the only difference between the QAAN and AAN architectures is the use of a QBM, rather than an RBM, we attribute to it the observed increase in performance. Nonetheless, QAAN’s learning advantage is not as substantial as QBM’s edge on simpler data sets, as supported by Amin et al. (2018); Kieferová and Wiebe (2017) and our results in Sec. IV.1. We suspect that this is due to the moderate size of the QBM used in our QAAN architecture. The bottleneck in our numerical experiments comes from simulating the QBM qubits on a classical computer through Monte Carlo sampling, which severely limits the accessible system sizes for the QAAN associative memory. Further improvements may be gained by improving our classical simulations and by considering QMA-hard Hamiltonians Kieferová and Wiebe (2017); Wiebe et al. (2019), which we leave to future work.

As QAANs are a quantum-assisted classical architecture, they lend themselves well to potential experimental implementations on NISQ devices. There are proposals for implementing QBMs on quantum annealing devices Amin (2015) and generic NISQ devices Anschuetz and Cao (2019), and the necessary Gibbs distributions have been produced on atomic lattice systems with similar Hamiltonians through Hamiltonian quenching Bernien et al. (2017); Kaufman et al. (2016). Furthermore, as the necessary number of visible units of the QBM only grows as the dimensionality of the latent space of the QAAN—which in general is much smaller than the size of the probability distribution being approximated—the necessary number of qubits also remains small. For instance, our simulations considered Boltzmann machines with $32$ visible units and $8$ hidden units (see Appendix C), whereas the dimensionality of the MNIST input data is $784$ and the dimensionality of the CIFAR-10 input data is $3072$. These considerations suggest that a QAAN could be implemented in the very near future.

While preparing this manuscript, we became aware of a similar project that also considered quantum-assisted classical AANs Wilson et al. (2019). We believe that our work, although similar in network architecture and scope, is still very different when it comes to model training and testing. In particular, we use quantum Monte Carlo to train the QBM, while the authors in Wilson et al. (2019) had access to a quantum annealing platform with many more qubits than our simulations could achieve. Nonetheless, our quantum-classical implementation seems to yield better generative performance under the considered metrics. Furthermore, we test our models on a more complex data set of colored images.

In conclusion, we have introduced a new hybrid quantum-classical generative model capable of successfully learning distributions over complex data sets. We have showed numerically that our model slightly outperforms analogous classical generative architectures when trained under similar conditions. In addition, the model could potentially be experimentally tested on NISQ devices. Our work adds to the rapidly-expanding family of quantum-enhanced machine learning algorithms.

Acknowledgements.

We are grateful to Maxwell Nye for insightful discussion on classical machine learning. We would also like to thank Aram Harrow for his guidance on this project. ERA is partially supported by a Francis C. W. Greenlaw fellowship, the Henry W. Kendall Fellowship Fund, and a Lester Wolfe fellowship from MIT. CZ is partially supported by a Whiteman fellowship from MIT.

Appendix A RBM Training and Sampling

We can explicitly write the gradients for each of the RBM parameters by substituting Eq. (2) into Eq. (5):

[TABLE]

Although the probability distribution over the visible and hidden units $P\left({\bm{v}},{\bm{h}};\bm{\theta}\right)$ is generally intractable for computing expectation values, the bipartite structure of the RBM makes it very simple to heuristically sample from the conditional probability distributions:

[TABLE]

where $\operatorname{\upsigma}\left(x\right)=\left(1+\exp\left(-x\right)\right)^{-1}$ is the sigmoid function. We can use these relations to approximate the negative phase of the gradient during training.

Two common algorithms used for this task are Contrastive Divergence (CD) Hinton (2002) and Persistent Contrastive Divergence (PCD) Tieleman (2008b). CD uses a data sample $\bm{v}_{\textrm{data}}$ as a starting point of a Markov chain at every optimization iteration and then performs block Gibbs sampling for $k$ steps using Eqs. (19) and (20). PCD instead relies on a single Markov chain with a persistent state that is preserved at the end of each optimization iteration and passed on as the initial state of the next iteration. In practice, choosing $k\in\left[1,10\right]=O\left(1\right)$ has been shown to be sufficient Hinton (2012).

In order to generate samples from the RBM, we apply the same procedure, but instead of initializing the visible units with a data sample, we use a random initialization. Furthermore, we use simulated annealing Kirkpatrick et al. (1983) by introducing an inverse temperature $\beta$ that scales the energy function, and hence the model parameters. We initialize our Markov chain with a sample drawn from a uniform distribution at $\beta=0$ and then gradually lower the temperature at each step of the Markov chain until we reach $\beta=1$, which corresponds to the desired distribution parameters. We use a linear schedule in $\beta$ for annealing. This procedure improves the diversity of our samples by increasing the mixing rate of the chain and helps the training procedure avoid getting trapped in local minima of the loss function.

Note that thus far we have assumed that our inputs—given by the visible units—are binary vectors. In the case of real-valued data, such as pixel values of images, we can often rescale the input to be in the range $\left[0,1\right]$ and treat it as the expectation value $p_{i}$ of a binary variable Goodfellow et al. (2016). We can then sample each entry $i$ in our visible vector $\bm{v}$ from a Bernoulli distribution with mean $p_{i}$. It is important to mention that although there are formulations of RBMs with real-valued inputs, known as Gaussian-Bernoulli RBMs, here we use Bernoulli RBMs as they are generally easier to train Hinton (2012).

Appendix B QBM Training and Sampling

We begin by rewriting Eq. (14) for each of the QBM parameters, such that at every training step we must estimate:

[TABLE]

As mentioned in Sec. (III.1), the use of the approximate lower bound given by Eq. (12) precludes the training of $\bm{\varGamma}$. The positive phases of the gradient can be calculated exactly, as given in Amin et al. (2018):

[TABLE]

where

[TABLE]

As in the RBM case, it is difficult to estimate the negative phase of the gradient as it requires sampling from a quantum thermal state, a problem which in general is NP-hard Barahona (1982). To approximately sample from our numerically simulated QBM, we perform population-annealed Monte Carlo sampling. In the population annealing sampling heuristic Hukushima and Iba (2003), a population of $R_{0}$ replicas of the system in question is maintained at infinite temperature, and then cooled to some finite temperature by an annealing schedule of $l$ steps, which is analogous to the number of Gibbs steps $k$ in the training of RBMs. With each cooling step, replicas are duplicated or deleted based on an estimate of their relative Boltzmann weights, and are equilibrated according to some Monte Carlo algorithm Machta (2010). By sampling a population from our quantum Boltzmann distribution, we are able to parallelize the generation of a mini-batch of samples with one run of the algorithm.

To perform Monte Carlo sampling at each equilibration step, we use the Trotter–Suzuki mapping of the stoquastic Hamiltonian to a classical energy function with an extra imaginary time dimension, which is discretized into $M$ imaginary time slices Suzuki (1976). Under this mapping, the quantum thermal distribution at inverse temperature $\beta$ given by the stoquastic Hamiltonian is approximated by the classical thermal distribution:

[TABLE]

where

[TABLE]

We impose periodic boundary conditions, such that $m=M+1$ is identified with $m=1$. In the limit $M\to\infty$, this approximation is exact. Then, we perform the necessary Monte Carlo sampling using the Metropolis–Hastings algorithm Hastings (1970) on the mapped set of spins.

In our simulations, we use $R_{0}=64$ initial population replicas corresponding to a single mini-batch, a linear annealing schedule of $l=5$ steps from $\beta=0$ to $\beta=1$, and $M=10$ imaginary time slices.

Appendix C Training Parameters

We train all of our networks using the Adam method for stochastic optimization Kingma and Ba (2014), with $\beta_{1}=0.5$ for all trained variables, $\beta_{2}=0.9$ for our Boltzmann machines, and $\beta_{2}=0.999$ for our generator and discriminator. As in previous works involving QBMs Amin et al. (2018); Khoshaman et al. (2018), we take all $\varGamma_{a}=2$. During training on synthetic data, we set the learning rate for both of our Boltzmann machines to $10^{-3}$ and use $k=5$ Gibbs steps. Each Boltzmann machine has $8$ visible units and $2$ hidden units, which are sufficient for approximating the studied Bernoulli distribution.

When training on the MNIST and CIFAR-10 data sets, we consider Boltzmann machines with $32$ visible units and $8$ hidden units. For consistency, we also set the dimension of the noise distribution for DCGAN to be $32$. We use a learning rate of $2\times 10^{-4}$ for both our generator and discriminator, and a learning rate of $10^{-3}$ for our Boltzmann machines with $k=5$ Gibbs steps. Furthermore, we initialize our weights using Xavier initialization Glorot and Bengio (2010) and initialize our biases to zero. In order to help the discriminator learn in the early stages of trainig, we use soft and noisy labels where a random number between [math] and $0.1$ is used instead of [math] labels (fake images) and a random number between $0.9$ and $1$ is used instead of $1$ labels (real images). Each model is trained for $30$ epochs, where an epoch represents one full pass through the training data.

Appendix D Performance Evaluation Metrics

In this section we describe how various GAN architectures are quantitatively compared. For data sets drawn from a known distribution $p_{\textrm{data}}$, we can evaluate the performance of a GAN by computing the KL divergence between the empirical distribution of generated samples and $p_{\textrm{data}}$. Unfortunately, for most image data sets the underlying distribution is unknown and this method is not applicable. Ideally, one would want humans to judge the quality of generated samples in comparison with the original data. However, as this approach is very subjective and almost always impractical, two alternative metrics have been proposed and successfully used to evaluate GANs.

D.1 Inception Score

Salimans et al. Salimans et al. (2016) proposed the use of a pre-trained neural network—the Inception-v3 network Szegedy et al. (2016) trained on the ImageNet data set Deng et al. (2009)—to assess the quality of the generated images. This metric is called the Inception score Salimans et al. (2016) and is defined as the average KL divergence between the conditional label distribution $p\left(y\>|\>\bm{x}\right)$ and the marginal distribution $p\left(y\right)$ over generated samples; that is:

[TABLE]

where $S$ denotes the entropy and the expectation value $\expectationvalue{\cdot}_{\bm{x}}$ is taken over image vectors $\bm{x}$ sampled from $\mathcal{G}$. We recognize the term on the right-hand side as the exponential of the mutual information $I(y;\bm{x})$. This metric captures two key features that we are looking for in our generated images: the depiction of clearly identifiable objects (i.e. $p\left(y\>|\>\bm{x}\right)$ should have low entropy for easily classifiable samples) and a high diversity in samples (i.e. $p\left(y\right)$ should have high entropy if all classes are approximately equally represented) Barratt and Sharma (2018).

Note that the use of the Inception network in computing the above score is only appropriate for colored images, which is not the case for the MNIST data set. We follow the approach described in Kodali et al. (2018) and train a simple 4-layer CNN on MNIST, which achieves an accuracy of $99\%$ and therefore can be viewed as a reliable classifier. We then use it to compute the Inception score in an otherwise identical fashion.

The Inception score is a widely adopted evaluation scheme and is known to match well with the human perception of image quality Salimans et al. (2016). However, it also has some known drawbacks, such as favoring models that memorize the training data or those that generate clear yet unnatural combinations of objects Borji (2018); Barratt and Sharma (2018).

D.2 Fréchet Inception Distance

The Fréchet Inception distance, introduced by Heusel et al. Heusel et al. (2017), avoids some of the problems of the Inception score described above by directly comparing the statistics of synthetic samples to those of real world data. This metric computes the similarity between the features extracted by the pool3 layer of Inception-v3 when the network is supplied with real data and with images generated from $\mathcal{G}$. These features can be thought of as drawn from multivariate Gaussian distributions $\mathcal{N}\left(\bm{\mu}_{\textrm{data}},\bm{\varSigma}_{\textrm{data}}\right)$ and $\mathcal{N}\left(\bm{\mu}_{\mathcal{G}},\bm{\varSigma}_{\mathcal{G}}\right)$ respectively. The Fréchet distance, also known as the Wasserstein-2 distance, is defined as:

[TABLE]

This distance is lower when the features extracted from generated data are distributed similarly to those extracted from real images.