Algorithms Clearly Beat Gamers at Quantum Moves. A Verification

TL;DR

This paper demonstrates that simple, well-implemented algorithms like GRAPE and coordinate ascent outperform human players and previously claimed algorithms in solving the Quantum Moves game, challenging earlier assertions about human intuition's advantage.

Contribution

The paper verifies that basic algorithms outperform both players and complex algorithms derived from player solutions, contradicting prior claims about human intuition's superiority.

Findings

Coordinate ascent outperforms all players and algorithms based on player solutions.

GRAPE significantly outperforms human players and matches or exceeds other algorithms.

Previous claims about human intuition being crucial are refuted by these results.

Abstract

The paper [S{\o}rensen et al., Nature 532] considers how human players compare to algorithms for solving the Quantum Moves game BringHomeWater and design new algorithms based on the intuition extracted from players. The claim by [S{\o}rensen et al., Nature 532] is that players outperform widely used algorithms, in particular the KASS algorithm, based on the Krotov algorithm, and that player intuition is crucial to develop improved methods. However, as initially discussed by D. Sels [D. Sels, Phys. Rev. A 97], a standard Coordinate Ascent algorithm outperforms all players by a large margin. Albeit D. Sels only compare to player solutions, the simple algorithm outperforms all algorithms based on player solutions and Krotov, and it does so using much less time and iterations. In this paper we elaborate on the methods discussed by D. Sels and verify that the presented algorithm, solves the problem better than all players and algorithms derived from player solutions in [S{\o}rensen et al., Nature 532]. We also verify the theoretical analysis presented by D. Sels, that gives a theoretically derived protocol that outperforms all players. We add a comparison with gradient ascent or GRAPE. Starting from uniform random values, GRAPE outperforms all players by a large margin. GRAPE works at least as well as the methods from [S{\o}rensen et al., Nature 532] initialized with player solutions. A standard analysis of the results from GRAPE provides a starting point for GRAPE, that outperform all algorithms from [S{\o}rensen et al., Nature 532]. We compare with a basic Krotov algorithm, and get results similar to GRAPE, clearly outperforming players and the KASS algorithm. These experiments verify and underline the result in [D. Sels, Phys. Rev. A 97] that the conclusions from [S{\o}rensen et al., Nature 532] regarding algorithms are untenable. In fact the opposite conclusions are true.

Figures

c of [[1](#bib.bib1)]). In Figure [13](#S5.F13), we show the excitation for the same experiments as in Figure [12](#S5.F12). The analytical protocol derived by D. Sels uses counterdiabatic driving and moves to an accelerated frame to solve the problem. In the new frame Sels looks for the ground state. However, once this is transformed to the original frame, the atom does not stay in the ground state. This can be seen in Figure [13](#S5.F13) very clearly. Here we see that the CD protocols are very similar to the numerical protocols with SA and GRAPE. This confirms once again that the theoretica

Tables

Table 1. (a) Fidelity Trace, duration 0.1, 40 steps, 128 allowed positions for the controllable tweezer, 100 traces as in [ 4 ] . Fidelity range of experiment, 0.533 - 0.537

Table 2. (a) Grape Position and Amplitude Heat Map for duration 0.1, using 40 steps

Equations

$$F(P)=|\text{matrixelement}\varphi U_P\psi {|}^2$$

$$u_i(k{\delta }_t)=w_i(k{\delta }_t)+\sum _{n=1}^{N-1}X[n]\sin \left(\frac{n\pi k}{N}\right)$$

$$H=p^2/2m-B{e}^{-(x-x_{\text{start}}{)}^2/(2\sigma )}$$

$$H=p^2/2m-B{e}^{-(x-x_{\text{end}}{)}^2/(2\sigma )}$$

$$H_t=p^2/2m-A_t{e}^{-(x-x_t{)}^2/2\sigma }-B{e}^{(x-x_{\text{start}}{)}^2/2\sigma }$$

$$t_{\text{mat}}=\frac{0.5}{d_h^2}\left[\begin{array}{cccccc}2& -1& 0& & 0& 0\\ -1& \ddots & \ddots & \ddots & & 0\\ 0& \ddots & \ddots & \ddots & \ddots & \\ & \ddots & \ddots & \ddots & \ddots & 0\\ 0& & \ddots & \ddots & \ddots & -1\\ 0& 0& & 0& -1& 2\end{array}\right]$$

$$\text{diag}(\exp (-((g_h-x_{\text{start}}{)}^2)/(2{\sigma }^2))$$

$$\text{diag}(A\exp (-(g_h-t_p{)}^2/(2{\sigma }^2)))$$

$$h_p=t_{\text{mat}}-M_b-M_p$$

$$U_p=\exp (h_p\cdot (T/N)(-i)).$$

Full text

1 Introduction

The paper [1] presents and discusses results for a Quantum Moves game, BringHomeWater, where players have attempted to move a quantum state from one position to another in a simulated optical tweezers and atoms setup. The paper compares the player solutions to a specific numerical method the authors have tested. In particular, [1] show evidence that an algorithm, denoted KASS, based on the Krotov optimization method [5] performs worse than solutions that the human players have come up with. Given the assumption that the Krotov method has been correctly applied, the evidence points to the fact that human players can outperform the Krotov method. This is the find and claim of [1] and it features prominent in the abstract of the paper. It leads the authors to conclude that “players succeed where purely numerical optimization fails, and analysis of their solutions provide insights into the problem of optimization of a more profound and general nature.” While it seems clear from the presented data that human players have indeed outperformed a particular implementation of the Krotov algorithm (and, according to [1], as a consequence also algorithms like CRAB [6] and differential evolution approaches [8], which is stated to perform worse than Krotov), this finding is of no significance. In fact, as has been discussed first by D. Sels [4], a very simple approach using classical arguments can capture the BringHomeWater Quantum Moves game far better than the player approach. Furthermore, as also shown by [4], one of the simplest optimization algorithms available, Stochastic Ascent, can outperform all of the above. This clearly demonstrates that the problem solved in [1] is not hard at all and efficiently solvable using well-known numerical methods. The authors of [1] note in the abstract that ’Using player strategies, we have thus developed a few-parameter heuristic optimization method that efficiently outperforms the most prominent established numerical methods.’ This must be regarded as the major outcome of [1]. This is a completely misleading conclusion when a simple Stochastic Ascent algorithm can be used to solve the problem better than any of the methods discussed in [1].

In [1], an implementation of Krotov named KASS is prominently featured and stated as the best algorithm found among several algorithm tested in [1]. It has a reported calculation time of 6 hours per run (with 2400 runs reported this is 600 days of calculation) so the first observation is that this is an extremely slow method. It is much heavier in computing time than any of the algorithms discussed in [4, 9], and all the results we discuss. This leads us to conclude that the KASS method is seriously challenged by the problem at hand. Surprisingly, there is no discussion in [1] of the standard gradient ascent approach, GRAPE [7]. We show that GRAPE solves the Bring Home Water problem very efficiently, much better than players and KASS. Furthermore, we have implemented the Krotov algorithm and find that it performs similarly to GRAPE. It is therefore very puzzling how the KASS algorithm [1] appears to be so poor.

BringHomeWater

The BringHomeWater game is defined by a setup with two tweezers and an atom, and the job is to use the controllable tweezer to move the atom, initially caught by the other tweezer, to some fixed position. The tweezer that has initially caught the atom cannot be controlled and does not change i.e. it does not move and is turned on at a fixed amplitude. The other tweezer is controllable by the user/algorithm, and may be move around and its amplitude is controllable as well. The job of a player or an algorithm is to use the controllable tweezer to move the atom from its initial position to a given target position, intuitively by moving the controllable tweezer towards the atom, and then transporting it back again to the target position, while ensuring that the atom does not escape the tweezer (spill over in BringHomeWater analogy) using as little time as possible. See [2] for more details.

We note that, surprisingly, the numerical optimization problem that is solved in BringHomeWater is not really defined in the paper [1], neither in the main text, methods, or supplementary materials. In particular several important parameter values are not given. A description and the parameters can be found in [4], and we follow the description given there. Initially, the atom and the fixed tweezer is placed at position $x_{\textrm{start}}=0.55$. The task is to move the atom to $x_{\textrm{end}}=-0.55$ using the controllable tweezer, that in the playable game is initially placed at $-0.55$ as well. The amplitude of the fixed tweezer is set to 130, and the controllable tweezer we allow to have an amplitude in the interval [0, 160].

The game is defined for any fixed duration of time to move the atom. The larger the duration the easier the problem, and the goal is to solve the problem with the smallest possible duration. Given a fixed time duration, a protocol for the controllable tweezer is a list of positions and corresponding amplitudes of the controllable tweezer for a set of time-steps. The quality of a protocol is determined by the so called fidelity under the condition that we have zero velocity of the atom at the start and end of the protocol. Denote the desired target state of the atom by $\bra{\phi}$ and the initial state as $\ket{\psi}$. For a given protocol $P$, the state of the atom after the protocol is given by the time evolution operator $U_{P}$ applied to the initial state. The fidelity of the protocol $P$ is formally defined as

[TABLE]

which is what we want to maximize over choice of protocol $P$. To be able to compute the fidelity we need to compute the initial and target state vectors and the required evolution operator $U_{P}$. This is specified in Appendix A.

The fidelity dictates whether the atom has been correctly moved to the target state and the goal is to find solutions with a fidelity of at least 0.999 [1]. As in [1] we define the QSL of an algorithm as the smallest duration for which the algorithm has returned a protocol with fidelity 0.999. This gives one number to easily compare different methods for the problem.

Stochastic Ascent:

To make the point clear that no elaborate algorithms or initialization is required for BringHomeWater, the Stochastic Ascent algorithm that solves BringHomeWater is given in Figure 1.

This is a simple and straightforward Coordinate Ascent [3] optimization algorithm from mathematical optimization. The only thing that may be considered complicated, is the computation of the fidelity, but that has nothing do to with the algorithm itself. In words, the algorithm is as follows. Let us say we are given 40 time steps to move the atom. Then we have 40 pairs of parameters, the position and amplitude, of the controllable tweezer at each of these steps. The task is now to find a set of parameters that makes the fidelity large. First, initialize randomly the parameters. That will almost surely give a bad fidelity score. Now, repeatedly iterate over the parameters in random order. When the algorithm consider a parameter, the remaining 39 are fixed. The algorithm then picks the value, pair for the parameter considered, such that in combination with the fixed parameters, the fidelity is as large as possible. Finally, these these random scans of the parameters are repeated until the fidelity stops improving. You could visualize this as having 40 knobs you can turn up and down, and every time you turn one the score changes. The algorithm then simply randomly selects knobs to turn at random, and turns the knob chosen to the position maximizing the score with the remaining knobs fixed. To solve the problem of computing the best value for a given parameter, D. Sels [4] simply impose a uniformly spaced fixed grid of allowed values for the parameters (knobs) and then simply tests all possibilities in the algorithm, instead of elaborately trying to find the exact best position, amplitude pairing.

Our Results:

In this paper we analyze the results from [1] more thoroughly and verify the results presented by D. Sels [4], and then add comparison to basic GRAPE and Krotov algorithms. We experimentally verify the results achieved with stochastic ascent, by making our own implementation of the algorithm and performing the same experiments as in [4]. We also test a few variations of the algorithm. Our experiments match and verify the experiments given in [4] that show that this simple algorithm easily outperform all player solutions and algorithms from [1], and that it is indeed very fast, much faster than the algorithms presented in [1]. We also experimentally test the theoretical work performed by D. Sels [4] and verify that the fidelity reported for the theoretically derived protocol are correct. We then compare the excitation of the atom to be transported in the theoretically derived protocol to the excitation of the atom caused by the protocol found by stochastic ascent and show that they actually look quite similar at the low energy levels. Finally, we use the theoretically derived protocol as a starting point for the GRAPE algorithm and this combination easily outperform all players and all algorithms developed in [1] in one try. This implies that the theoretically derived counterdiabatic approximate protocol is in fact quite good.

Now Stochastic Ascent is a standard Coordinate Ascent algorithm algorithm. It is, however, not as famous as as the gradient ascent algorithm. To add perspective to the algorithmic part of the problem, we test the more famous gradient ascent algorithm which in Quantum Control is know as the GRAPE algorithm described in 2005 [7]. First, we initialize GRAPE with completely uniform values for both positions and amplitudes. With this basic setup GRAPE clearly outperforms all players and the KASS algorithm. This is highly surprising since gradient ascent is perhaps the most famous algorithm in mathematical optimization and has been considered in Quantum Control as the GRAPE algorithm for more than a decade. Furthermore, we started the algorithm from the most naive starting positions and amplitudes imaginable. By visualizing the positions and amplitudes output by the GRAPE algorithm with heat maps, we extract new starting positions and amplitudes which when used with the GRAPE algorithm, clearly outperforms all players and all algorithms developed in [1]. We continue to investigate the random starting positions and amplitudes used in the KASS algorithm. Since the KASS algorithm is not fully specified in [1] we do a best approximation. We run the GRAPE algorithm from these random starting positions and amplitudes and show that the GRAPE algorithm with these initial values also outperforms all human players and the actual KASS algorithm by a wide margin.

While the KASS algorithm is a special version of the Krotov algorithm with specific seeds and sweeps, it is surprising that the Krotov algorithm, a standard algorithm in Quantum Control, performs so much worse than GRAPE and Stochastic Ascent. We implement a standard Krotov method and apply it in the same way as GRAPE with completely random starting positions and amplitudes. The results for our Krotov implementation resemble those achieved with GRAPE, and easily outperform all players and the KASS algorithm. Comparing to [1], both our GRAPE and Krotov implementation take only a thousand iterations over the parameters to converge, nothing like the more than 100 million trials as stated in [1].

We have shown the results achieved for different durations by our standard implementation of standard algorithms in Figure 2. For each duration the goal is to have as high as fidelity as possible and the longer the duration the easier the problem. Thus the goal is to be close to the left top corner. As we can see from the figure all the algorithms behave fairly similar except for the KASS algorithm. It is extremely poor when compared with the other basic algorithms, including our implementation of the Krotov algorithm. This proves that the players do not succeed where algorithms fail nor do player inspired solution outperform even basic algorithms in any way, which was the main conclusions from [1]. In fact the figure show the opposite to be true.

Outline

In Section 2 we present all our experiments with stochastic ascent. In Section 3 we present our experiments and results for the GRAPE algorithm, and in Section 4 we show our experimental results for the Krotov algorithm. In Section 5 we provide our experiments with the theoretical derived protocol from [4]. Finally, we summarize in Section 6. The appendix contains all the needed derivations for our algorithms and code for Stochastic Ascent, GRAPE and Krotov is available at GitHub.

2 Stochastic Ascent Experiments

In this section we experimentally show that the Stochastic Ascent algorithm as presented in [4] is far superior to human players and the algorithms presented in [1] that use player solutions as initial seeds for other algorithms.

In all experiments, the amplitude of the controllable tweezer is set to 160 if nothing else is stated following [4]. Extended Data Figure 3 of [1] indicates that an amplitude of $A\sim 150$ may be the best of highest value for the amplitude of the controllable tweezer since this is the maximal value used by the algorithms in [1], that states in regards to KASS that ‘Optimizations showed that $A=-150$ was the optimal value’. This seems to mean that the 150 is the best value to use for the algorithms used in [1]. Here we use the values provided by [4] taken from the BringHomeWater game itself, i.e. we use the maximal allowable amplitude as provided to the gamers.

The reason for fixing the amplitude at the maximal value is based on two observations. First, the analytic results in [4] shows consistently better results when the amplitude is larger. Second, since it is a linear control field in the hamiltonian, Pontryagin theorem tells us that the solution is (almost) always on the boundary. This means that $A=160$ or $A=0$ at any time and you can switch between these two. But since $A=0$ means that the atom is not being confined the most likely optimal solution is that $A=160$ all the time (besides maybe for the first step). For completeness and generality, we also tested adding more allowable amplitudes, which is simple extension of the algorithm, but that only made the algorithm slower without changing the results, supporting the theory and the choice in [4].

To cover the different possibilities we consider different versions of the Stochastic Ascent algorithm from [4]. We add a version where the first position of the controllable tweezer is always $-0.55$ as in the game, and a version where additionally the amplitude of the controllable tweezer is set to 150.

2.1 Full Comparison

In Figure 3 we have shown a comparison of the Stochastic Ascent algorithms with the players and the derived algorithm from [1]. For each duration $T$ we can compare the score of the different approaches, higher being better. This plot clearly shows the superiority of the simple Stochastic Ascent algorithm over all players by a massive margin. The gamers are not even close to the performance of the simple algorithm. Comparing with the algorithms derived by using player solutions and the KASS algorithm, the plot shows that these are also clearly beaten by the simple algorithm.

The difference between fixing the first position of the controllable tweezer at $-0.55$ and allowing the algorithm to optimize this step is very small and makes essentially no difference for the result. If we reduce the amplitude of the controllable tweezer to 150 the results are also slightly worse. This reduction in QSL is predicted by [4] that says QSL should follow $1/\sqrt{A}$ and the reduction we see in the experiments fits well with that. If we allow a higher $T/N$ ratio and and tune other parameters like the number of possible tweezer positions the QSL may possibly be reduced for all versions of the Stochastic Ascent algorithm. Since the results of the Stochastic Ascent are already clearly superior, and thus verifying [4], we do not follow this direction further in this note.

To experimentally verify that it is a good strategy to start by a fast move from the starting position at $-0.55$ to where the atom is caught by the fixed tweezer we have visualized a protocol for $T=0.16$ that achieves a fidelity above $0.999$.

2.2 Fidelity Traces

We have also created the fidelity trace plot from ([9] Figure 6), that zooms in on one the most volatile durations at $T=0.1$. We have created the plot for the two invocations of the algorithm as above, one invocation where the first position is optimized in the algorithm and one invocation where the first position is fixed at $-0.55$, see the figures in Table 1. A simple visual comparison shows that our implementation gets results that are essentially identical to the results reported in [9].

The simple conclusion is that the algorithm essentially gets the same fidelity all the time with different but very similar protocols having found local minima (in the sense that one parameter change cannot increase the score) with almost the same fidelity. As noted in [4] the algorithm converges after very few iterations over the parameters, each round taking only a few seconds on a laptop. Fixing the first position of the tweezer does not change this, only reduces the fidelity achieved slightly for this duration as was also apparent from Figure 3. It is not surprising that fixing the first position of the controllable tweezer does not change much, since fixing the first position essentially just corresponds to a slight perturbation of the starting state from $\ket{\phi}$ to $U_{-0.55}\ket{\psi}$, where $U_{-0.55}$ is the evolution operator the first step with the controllable tweezer at $-0.55$, and then running the algorithm from this configuration instead. The simple algorithm does not care about the starting and ending state, and if we add the fact that we could have used the smallest possible amplitude allowed for the controllable tweezer for that position of the protocol the change in the starting state would be even smaller. In the extreme case of the amplitude of the controllable tweezer being zero at the first step the only force acting on the atom is the fixed tweezer that already has the atom caught. The difference in fidelity between the two algorithms in Table 1 is around 0.04. Running Stochastic Ascent while not fixing the first position for a duration $T=0.0975$ and $N=39$ (maintain the same $T/N$ trade off) the fidelity achieved hovers closely around 0.492, essentially the same as for Stochastic Ascent with fixed first position with one additional step and time unit. As the fidelity achievable varies greatly around $T=0.1$ losing a step means losing $T/N$ time which matches the difference between the two versions explaining the discrepancy. Since we have explained the difference we see no reason to include an experiment with a fixed starting point for the controllable tweezer at $-0.55$ while allowing amplitude of zero. We conclude that our experiments are in full agreement with the results reported by D. Sels [4] verifying his results. Note that the protocols that are found by both version of the algorithms are the simple intuitive ones as above: Move the tweezer to the atom, move the tweezer back slowly, and shake it on the way to avoid spilling.

2.3 Superposition states

As discussed in [4], the BringHomeWater (Quantum Moves) may not be the best representation of quantum problems. To add as much quantum complexity as possible to the BringHomeWater challenge, [4] suggests replacing the target state $\bra{\phi}$ with a superposition of the target state and end the initial state, i.e. $\bra{\phi}_{q_{1},q_{2}}=q_{1}\bra{\phi}+q_{2}\bra{\psi}$. In Table 2 we have the fidelity achieved for different durations and a representative high fidelity protocol for a specific superposition. Now this protocol, looks much more complex than in the “vanilla” BringHomeWater and it would be fun to see if human players can get high fidelity here (as the simple stochastic ascent algorithm can).

The Stochastic Ascent algorithm is able to solve the superposition version of the problem as well, albeit as noted in [4] the variance of the result of a run is higher for this problem than it was for the vanilla BringHomeWater, and more runs of the algorithm may be required to achieve high enough fidelity. This indicates that this is more complex optimization problem.

Together with the previous experiments this shows that the simple algorithm efficiently solves the BringHomeWater game in any configuration without any prior information with better performance than all results provided in [1].

3 Gradient Ascent Algorithms

Seeing that a standard algorithm from mathematical optimization completely outperforms all players and the best results in [1] that has used more than ten thousand computer hours raises many questions. For instance, how poor is this KASS algorithm which is stated to be the best algorithm the authors found for BringHomeWater in [1]. Since we already know how to compute the fidelity of a given protocol we can straightforwardly implement a gradient ascent algorithm, which in quantum control is known as the GRAPE algorithm. For this case, GRAPE is no more than gradient ascent on the tweezer positions and amplitudes. So instead of fixing a grid of allowed positions and amplitudes and iteratively improving the cost by picking one variable pair to optimize, GRAPE instead computes the gradient of the fidelity of all positions and amplitudes and takes a step along that direction. As mentioned earlier, this algorithm is surprisingly not mentioned [1] despite that it appears that it is a standard algorithm in quantum control [7]. The implementation details can be found in Appendix A, in short we implement the fidelity computation in TensorFlow and use automatic differentiation.

We have not elaborately tested gradient ascent, since there are so many variants and tools to improve on the quality and convergence of the methods. We just want to check how a normal implementation compare with the other solutions. The main question we need to consider is how to initialize the protocol before starting the iterative algorithm. Gradient ascent requires that we give the algorithm a reasonable starting point. Finding a bad one is not hard, for instance fixing all the initial tweezer positions at the left end or something similar as a starting position gives essentially zero gradient to work with and the gradient ascent algorithm fails to do anything. Gradient ascent does not search the space of solutions in some clever way, the algorithms moves uphill until it dies down in a flat area, so if we start it at a poor place the results will be poor. No surprise there. To hinder the algorithm as much as possible, without killing it, we start by testing GRAPE using the classic and very poor strategy of assuming that we do not know anything about the problem and hence do uniform sampling of parameters in their range. This means that the positions are sampled uniformly at random from $[-1,1]$ and the amplitudes are chosen uniformly at random in $[0,160]$.

The results for the uniform initialization out-of-the-box GRAPE implementation is shown in Figure 6. From this figure it is obvious that GRAPE also drastically outperforms all players and the KASS algorithm by a wide margin. In fact, GRAPE with completely uniform initialization is as good or better than running the algorithm from [1] on the players seeds. An if we use the protocol found for 0.18 that gets a fidelity of 0.99, just shy of the mark, as input for duration 0.19, we get a fidelity of 0.999, making it clearly better than starting from player seeds with the algorithm in [1]. Still, this GRAPE with completely uniform initialization is slightly worse than the HILO algorithm derived from analyzing player solutions in [1].

3.1 Intrinsic Intuition of Gradient Ascent

Now if we do not wish to rely on any theoretical or intuitive understanding of the how to move the tweezer to get a better starting point than uniformly at random, we can instead analyze the protocols found by GRAPE. This is what the authors in [1] do with the player solutions to obtain their best algorithm. However, instead of trying an elaborate analysis of the protocols, applying different clustering and visualization techniques, we simple make heat map plots of the protocols, ignoring any dependencies between steps. The heat maps are constructed by splitting the position domain, $[-1,1]$, into 40 equal sized intervals, and for each step counting how many times a protocol has a position value in the each of the 40 intervals. We do the same for the amplitudes, using 32 buckets. We have shown the resulting heat map for duration 0.12 and 0.18 in Figure 7. In the Appendix we include the heat maps for the durations (0.1, 0.14, 0.16, 0.20) in Table 3. Inspecting the heat maps, the first thing that comes to mind is that the amplitude is almost always the maximal value (the top white bar), and in the few cases the amplitude does not converge to max, it is zero. Just like the theory suggested! The only difference is step one, where the controllable tweezer is far from the atom, here it seems that there is no preference and it has essentially no impact on the result. For the position of the controllable tweezer, the plots reveal the natural approach of moving the tweezer to the atom fast (in one step) and then slowly back again, showing that gradient ascent find this solution without knowing the game design.

It also clear from the heat-maps that from small durations (0.1-0.14) good GRAPE solutions all look almost exactly the same while for larger duration (0.16-0.18) it seems several solutions path exist among the best solutions found by GRAPE and as we increase the duration (0.20) most of the good solutions again look similar but different from the small duration protocols. Using this we create a new GRAPE experiment where we trace out the solution implied by the heat map for both amplitude and position. For the heat map solution for duration 0.16 we traced out the two most prominent paths and used as starting positions. The results is also shown in Table 6, where we can clearly see that with this initialization easily extracted from the heat map, outperforms all algorithms from [1]. In fact we get results just as good as we did for Stochastic Ascent albeit it took a little longer time to get there.

In conclusion, GRAPE with no information explores a much larger portion of the space because of the uniform initialization and still learns the intuitive protocols without access to first learning from simpler versions of the game nor the description or the graphical visualization of the task and gives better results than players and all algorithms from [1].

3.1.1 GRAPE with KASS Seeds

It is not hard to make local search algorithms like gradient ascent perform poorly if you provide bad initial starting points. If you happen to be in this regime, and cannot come up with any better initialization, one could argue that the the obvious solution is to not another algorithm. As we have shown above the most trivial of initialization work well, so it is strange that an algorithm like KASS with more elaborate initial values does so much worse. For this reason we decided to test this KASS algorithm a little more. We already know that applied in [1] it is pretty bad algorithm for the problem. But how does the initial starting position in KASS look like.

We quote from [1]:

We tried different seeding and optimization strategies, and the most successful was a Krotov algorithm using sinusoidal seed functions and sweeps over the total duration (KASS). KASS applies a sweep to an initial random seed $u_{i}(t)$ where $u_{1}(t)=A(t)$ is the tweezer amplitude and $u_{2}(t)=x(t)X$ is the tweezer position. The initial random seed is:

[TABLE]

where $N$ is the number of time slices in the path and $\delta_{t}=0.002$ is the time discretization. $w_{i}$ is the motion at constant speed to the initial position of the atom and back again for $A(T)=-100$. The amplitudes $X[n]$ were selected with a random sign and a norm decreasing as a function of the frequency.

There is no reference to where this construction comes from and as such it seems somewhat arbitrary. Furthermore, the starting values are not fully specified since $X$ is stated to be decreasing in relation to frequency, which could mean many many things, the range is not even specified. For the amplitudes as far as we understand, there is no constant motion but just the constant 100 and then the sinusoids. In conclusion we cannot test this seed structure and try and reproduce the results. What we test instead is simply to ignore the sinusoids and start from linear motion back and forth using half the time on each path as given by $w_{1}$ and then uniform random amplitudes in $[40,160]$. In this way the expected amplitude is 100 as it is for KASS as specified. Adding noise to the positions would only make the experiment look very close the GRAPE experiment already performed. We test these starting values with the GRAPE algorithm and get a QSL of 0.2, just as good as the best result starting from player solutions in [1]. The full comparison is shown in Figure 8 where we can see that we get much much better results than the KASS algorithm and also easily beat all the players. Albeit it gets the same QSL of 0.20 as GRAPE with uniform initialization, it deteriorates much faster for lower durations.

We have shown two heat maps from GRAPE-KASS in Figure 9. Here we can see that for high durations (0.18) the algorithm simply does not know how to transform the initial slow moving of the controllable consistently for instance to moving the tweezer fast to the atom as GRAPE and Stochastic Ascent found. Instead GRAPE-KASS often essentially removes the influence of these steps by moving the tweezer as far away from the atom as possible leaving the atom as it is. For the other half of moving the atom home, GRAPE-KASS has figured that our very well, including the shaking, and making the amplitude as large as possible. For the low duration the GRAPE-KASS algorithm does seem learn to move to the atom fast, but at the cost of moving the tweezer to the boundary in middle part of the protocol and not maximizing the amplitude in this part, again not using this interval of the protocol for controlling the atom very well. In conclusion this specific way of initializing the GRAPE algorithm, particularly the slow moving of the tweezer towards the atom seems to hinder the algorithm from finding high fidelity solutions for small durations and is generally worse than just uniform random sampling.

4 The Krotov Algorithm

In this Section we show the results for doing the same experimental setup we did for GRAPE using a standard Krotov implementation. As for GRAPE we initialize with completely uniform random positions and amplitudes and test the algorithm for different durations. Compared to the KASS algorithm this means we have removed the specific initialization used and the sweeps. A sweep just means the algorithm starts from high durations that are easy, find a good protocol for this duration and then down-sample and run on a slightly smaller duration and then repeating. The results are shown in Figure 10. As we can see from the figure, our simple Krotov implementation, without any help, easily outperforms all players and the KASS algorithm by a large margin obtaining almost the same result as we did with GRAPE. We have shown two heat maps from the Krotov runs in Figure 11 which reveal the same as the the heat maps for the GRAPE algorithm did, and as we can see from Figure 10 the heat map solution based on Krotov runs is as good as it was for GRAPE matching the results of stochastic ascent.

5 Experimentally Verifying Theoretical Derived Protocol

As we have noted earlier, there is an insignificant difference between the work of Sels [4, 9] and the work in [1] due to the first step of the protocols discussed by D. Sels. In [1] the controllable tweezer is fixed at the starting position in the first step, while in [4, 9] this positions is optimized by the method and placed very close to the atom instead, as keeping the first step fixed at $-0.55$ just wastes time. As we have amply shown, for Stochastic Ascent, this makes no real difference at all. In order to remove any doubt that the analytical protocols derived by Sels [4] using counterdiabatic driving (CD) indeed does capture the whole problem, we show in Figure 12 a comparison between the analytical protocols, the results using stochastic ascent (SA), and GRAPE initialized with the CD protocol. To ensure that there can be no issues for the comparison, we have resampled the CD protocol and fixed the starting point to -0.55. As we can see this does not really change the fidelity achieved by the protocol, just like it was the case for stochastic ascent. Indeed, we see in Figure 12 that there is a near-perfect match between the raw CD protocol and the resampled version and both match the fidelity reported for the protocol by D. Sels for the given time duration [4, 9]. It is also clear from the figure that the GRAPE algorithm initialized with this counterdiabatic protocol clearly beats all algorithm from [1] obtaining a QSL of 0.16.

We note that if we initialize the tweezer positions for GRAPE with the solution for a single tweezer version of the problem analyzed in [4] we get a QSL 0.16 matching the best results achieved. As described in [4] the simple line gives the smallest magnitude of the acceleration, which we need to avoid to avoid spilling. However, the line has the issue that the velocity does not cancel out at the beginning and the end. This is handled by using a third order polynomial instead in [4]. However, a gradient ascent algorithm may be able to take care of both these issues, and as the results shows it does. Starting with this simple initialization for gradient ascent by picking points on a line from end position to starting position gets a better QSL than the best algorithms from [1], actually obtaining the same QSL as the stochastic ascent algorithm and the Krotov and GRAPE heat map algorithms, in one try.

In the work [1], there is also a discussion of the excitation of the atom during the protocols (see Figure 3c of [1]). In Figure 13, we show the excitation for the same experiments as in Figure 12. The analytical protocol derived by D. Sels uses counterdiabatic driving and moves to an accelerated frame to solve the problem. In the new frame Sels looks for the ground state. However, once this is transformed to the original frame, the atom does not stay in the ground state. This can be seen in Figure 13 very clearly. Here we see that the CD protocols are very similar to the numerical protocols with SA and GRAPE. This confirms once again that the theoretically derived CD protocol discussed in [4] has correctly captured the problem. The difference between CD protocol and the numerical protocols, is due to the shaking motions that are seen in Figure 12 as well. However, the overall profiles are consistent. We may again conclude that the work in [4, 9], both theoretical protocols and numerical experiments, are correct, capture the whole problem, and show without a doubt that theoretical and numerical algorithms far outperform human players in the BringHomeWater game.

We have also tests a theoretically derived counterdiabatic protocol for duration 0.22. The analysis of the protocol for this duration shows the exact same thing and we leave it out.

6 Final Comments

We have experimentally verified the results of the simple Stochastic Ascent algorithm from [4]. We have tested different possibilities and shown that this does not change the performance of the simple Stochastic Ascent algorithm. We also experimentally tested the theoretically derived CD protocol from [4]. We conclude that the results, discussion and conclusions in [4] are verified. Furthermore, we have shown that a basic gradient ascent and a basic Krotov algorithm also easily outperform all players and the KASS algorithm. We note that the QSL of 0.16 achieved by different methods in this note, in particular stochastic ascent and gradient ascent and Krotov initialized properly may be improved for instance by testing slightly smaller durations.

As pointed out by D. Sels in [4], the player solutions are variations of the intuitive solution that move the tweezer to the atom and then slowly back again with a few shakes to avoid spilling. Using such protocols (while ignoring the shakes) gives essentially the initialization that seems to be the main result achieved in [1], albeit they use a different algorithm based on Krotovs method, that is worse than standard gradient ascent and basic Krotov, on top of it. Thousands and thousands of gamers have played the game but it seems clear that they have not come up with anything that, even when combined with the best algorithms by [1], is as good as the basic stochastic ascent algorithm or the basic gradient ascent or krotov algorithm initialized from a simple heat map analysis of several GRAPE or Krotov runs. The algorithms also reveal two similar but different strategies for solving the problem; one for short durations and one for larger. We note that we only tested basic algorithms from mathematical optimization and quantum control, and that there are many other more elaborate algorithms designed for hard computational problems that almost certainly also outperform both gamers and the player inspired algorithms from [1] as well. To summarize, all the conclusions made in [1] regarding algorithms and optimization is entirely based on poor application of algorithms. From the results it is clear that human players and human intuition with the help provided by the game designers is clearly outperformed by theory and algorithms. This suggests that this will also be the case for more complicated problems like for instance the superposition version of BringHomeWater as introduced in [4]. KASS was claimed to best numerical algorithm for Bring Home Water by [1], which seemingly lead to the conclusion that gamers beat algorithms. As shown by D. Sels in [4] and verified in this note, this is clearly not true, gamers do not even come close the performance of standard algorithms. In fact the KASS algorithm is exceptionally poor when compared with standard implementation of gradient ascent, and the Krotov algorithm which KASS is based on. In conclusion it is clear that that KASS implementation is a very poor algorithm for the problem. Thus the fact that players are able to beat KASS on a few occasions has no significance. Stated differently, the conclusion from [1] that “Players succeed where purely numerical optimization fails is false.

The other main result from [1] that “Using player strategies, we have thus developed a few-parameter heuristic optimization method that efficiently outperforms the most prominent established numerical methods”. As argued, this can only be true if you define the most prominent established numerical algorithms as KASS and CRAB, and as such is completely misleading. As shown by D. Sels in [4] and underlined by the results in this paper, this is clearly false as well. In particular, stochastic ascent is simpler, it is more efficient, it does not require hundreds of trials to get the a good solution and it achieves better results. The same holds true for GRAPE and Krotov. While the best method in [1] is better than uniform initialization GRAPE, a reasonably initialized GRAPE and Krotov easily beats it, and a good initialization can be made from theory or a standard analysis of protocols extracted from a few runs of the algorithm taking a few hours of computer time on a standard work station.

On a side note the results in [4, 9] and this note also contradict the argument from [1], that “The choice of an initial seed is a central challenge in such complex optimizations” as BringHomeWater and the statement that We stress that any optimal control effort requires a substantial amount of optimizations of different seeds to find good solutions., since no fancy initialization, using human players or otherwise, is needed.

Appendix A Stochastic Ascent Algorithm

The Hamiltonian’s defining the system in BringHomeWater is defined as follows. Let $B$ be the amplitude of the fixed tweezer, then the initial state of the atom is the ground state of the Hamiltonian

[TABLE]

and the final state is the ground state of the Hamiltonian

[TABLE]

Let $A_{t}$ be the amplitude of the controllable tweezer, and $x_{t}$ be the position of the controllable tweezer at time $t$ then the time evolution operator Hamiltonian is

[TABLE]

Following [4], we implement a Stochastic Coordinate Ascent algorithm for BringHomeWater as shown in Figure 1. As in [4], the algorithm does not optimize over the amplitude of the controllable tweezer, keeping the amplitude fixed at the maximum allowed value all the time as discussed earlier. Secondly, the algorithm uses a reasonably small discrete set of positions for the controllable tweezer which is simply a uniformly spaced grid of some fixed size $s$. Note that the same grid is used for all steps of the protocol. This allows computing all the needed unitary matrices only once before the iteration starts.

Now consider the configuration space of the problem. Let s denote the number of possible positions for the controllable tweezer. In any protocol we have to specify a number of steps that we can take, denote this by $N$. The configuration space is therefore $s^{N}$. For any reasonable values of $s$ and $N$, $s^{N}$ will be an astronomical number, and testing all protocols is not feasible.

A.1 Numerical Computations

To compute the fidelity of a protocol, we use a grid of $h_{g}$ of uniformly spaced points to discretize the Schrodinger equation and Finite Difference approximation for the Laplacian. This requires computing matrices for the three parts in the Schrödinger Equation (Equation 1). First, define a uniformly spaced grid $g_{h}$ over $[-1,1]$ with $h_{g}$ uniformly separated points (including -1, +1) at distance $d_{h}=2/(h_{g}-1)$. For the Laplacian, generate a finite difference matrix as $\frac{0.5}{d_{h}^{2}}$ times a tridiagonal matrix with two on diagonal and -1 above and below the diagonal.

[TABLE]

For the part of the Schrodinger Equation for the fixed tweezer, $Be^{-(x-x_{\textrm{start}})^{2}/2\sigma}$, generate a diagonal matrix $M_{B}$ where the ith entry is $Be^{-(h_{g}[i]-x_{\textrm{start}})^{2}/(2\sigma)}$ In vectorized notation this can be written as

[TABLE]

where $g_{h}$ is the list of grid points used for discretization.

For the final part of the equation for the controllable tweezer, $Ae^{-(x-x_{\textrm{tweezer}})^{2}/2\sigma}$, compute a matrix for each allowed position of the tweezer and for each such position, construct a matrix as for the fixed tweezer. Thus, for each tweezer position $t_{p}$, make a diagonal matrix where the $i$’th entry is $Ae^{-(t_{p}-g_{h}[i])^{2}/(2\sigma)}$. For position $t_{p}$ this can be written in vectorized notation as

[TABLE]

where again $g_{h}$ is the list of grid points used for discretization.

Finally, for every potential tweezer position $t_{p}$ with associated matrix $M_{p}$ compute the Hamiltonian matrix

[TABLE]

, and to get the unitaries compute

[TABLE]

This defines all the required unitary time evolution matrices, and for a given protocol the time evolution operator is the a product of an the corresponding multi-set of these ordered by the protocol. For the initial and target state compute the eigenvector with the smallest eigenvalue of the appropriate corresponding Hamiltonian matrix. In particular the starting state $\ket{\psi}$ is the eigenvector corresponding to the smallest eigenvalue of $t_{\textrm{mat}}-M_{b}$. For the target state, create the matrix $\bar{M_{b}}$ symmetric to $M_{b}$ just for $x_{\textrm{end}}$ instead of $x_{\textrm{start}}$, i.e. a diagonal matrix with the $i$’th entry equal to $Be^{-(h_{g}[i]-x_{\textrm{end}})^{2}/(2\sigma)}$. Now $\bra{\phi}$ is the eigenvector corresponding to the smallest eigenvalue of the matrix $t_{\textrm{mat}}-\bar{M_{b}}$. Notice that the controllable tweezer is turned off for both the start and end state.

A.2 Implementing Algorithm

With the math behind us the the Stochastic Ascent algorithm can now be implemented with a fairly short python program using numpy and scipy(or any other language if preferred). We always use $m=1,B=130,L=1.1,\sigma=\frac{1}{8}$ as defined in [4]. The remaining parameters depends on the experiment. A description of the algorithm is also contained in [4] but for completeness we include (a shorter) one here in Figure 14.

In [4], $s$, the number of positions available for the controllable tweezer is set to 128. With 128 as the number of tweezer positions we cannot even place the controllable tweezer at the atom or at the target position! For the general result of showing that the Stochastic Ascent algorithm easily and clearly beats human gamers 128 is more than enough. If it is important to get fidelity above 0.999 as the goal set in [1] it helps to increase this parameter for instance doubling it or setting it to 201 to ensure the controllable tweezer can be placed on $x_{\textrm{start}}$ and $x_{\textrm{end}}$ if needed. This will also of course make the algorithm slower by a constant factor. The grid size $h_{g}$ for discretizing the Hamiltonians is a numerical precision parameter which we set to 512, albeit for fast tests using a smaller value gives almost the same results. We checked that the discretization indeed changes very little to the results at that point. To fix the first position of the controllable tweezer in the protocol, initialize $x_{1}$ to $-0.55$ (or closest grid point) and then do not optimize that position in the algorithm by not including it in the random permutation $I$. To allow more amplitudes than $A=160$, we add the necessary unitary matrices as described above, and for each step optimize over position, amplitude pairs, scaling the number of possibilites to check with the discretization of the amplitudes considered. We have only tested this with a few parameter settings. The results from these experiments indicate that the algorithm naturally slows down but still finds solutions of the same quality, and as the theory suggest using the maximal value for the amplitude works.

A.3 Gradient Ascent - GRAPE

To implement gradient ascent, we implement the fidelity computation, as described above for Stochastic Ascent, in TensorFlow and use automatic differentiation. The number of steps, $N$, for a given duration, $T$, is set as for stochastic ascent such that $T/N=0.0025$. For each duration $T$ tested we ran several GRAPE invocations from completely uniform random starting points. Compared to Stochastic Ascent, GRAPE is more time consuming and in order to save time, we ran the main bulk of experiments using imprecise computation by simply setting a low value for the grid size $h_{g}$ and running each experiment only a limited amount of iterations. We ran 2000 experiments starting with, $h_{g}=32$ and ran it for at most 2000 iterations or until it had stopped improving (gradient to small). Then the best protocol found like this is considered again, we increase the grid size back by doubling and restarting the iteration, until we hit a grid size of 512. While there is no guarantee that best model for the small $h_{g}$ is the best model to continue from, it seems a reasonable choice and small tests confirm that this, and saves a lot of computing time. Since ADAM [10] is the standard recommended optimizer in TensorFlow (albeit for deep learning) we use that for the experiments and changed only the standard learning rate parameter. We start with an initial learning rate of 0.1, or 0.01 and reduce it during the algorithm when the fidelity stops improving. It takes around four to six hours on a standard work station to run all 2000 random starts and optimizing the best found protocol from there takes even less. For a full test one should really test different optimization methods and very their hyper parameters. Since such a large experimental evaluation is not the point of that note, we leave that to the interested reader. Since we only tested a few things we believe that even better results can be achieved for GRAPE with more experiments and tuning.

To ensure the tweezer starts as in the game we always fix the first of the $N$ position available to $-0.55$ and never update it in the algorithm.

A.4 Krotov Algorithm

To implement the Krotov algorithm, we use the already existing implementation of GRAPE for computing Hamilton’s and unitaries and base our Tensorflow implementation on that. We do not use automated differentiation for the Krotov algorithm as it turns out to be simple to write out the necessary gradient computations. The number of steps, $N$, for a given duration, $T$, is set, as for stochastic ascent and GRAPE, such that $T/N=0.0025$. For each duration $T$ tested we ran several Krotov invocations from completely uniform random starting positions and amplitudes. As for stochastic ascent and GRAPE, in order to save time, we ran the main bulk of experiments using imprecise computation by simply setting a low value for the grid size $h_{g}$ and running each experiment only a limited amount of iterations. We ran 1000 experiments starting with, $h_{g}=32$ and ran it for at most 1500 iterations or until it had stopped improving. Then the best protocol found like this is considered again, we increase the grid size back by doubling and restarting the iteration, until we hit a grid size of 512. As for GRAPE, it is important to handle the learning rates properly, and as for GRAPE, we again use the ADAM [10] algorithm for that , using a separate optimizer for positions and amplitudes. For the positions, we start with an initial learning rate of 0.01 and reduce it during the algorithm when the fidelity stops improving. Similarly, for the amplitudes, except that we start with a learning rate factor of 0.1. The reason for making this learning rate higher for amplitudes is simply that the gradient of the positions include the amplitudes as a scale factor, making them much larger than the position derivatives in absolute value. The same thing could have been added to our GRAPE implementation. It takes a few hours on a standard work station to run all 1000 random starts for a given duration and optimizing the best found protocol from there takes even less. Again, for a full test one should really test different optimization methods and very their hyper-parameters. To ensure the tweezer starts as in the game we always fix the first position of the protocol to $-0.55$ and never update it in the algorithm.

A.5 Actual Code

Code for running the algorithms and notebooks that shows how to run the algorithms by example can be found at: https://github.com/gronlund/QuantumMoves.