Distributed Black-Box Optimization via Error Correcting Codes

TL;DR

This paper presents a distributed black-box optimization method using error correcting codes, improving resilience to stragglers and accelerating computation, especially for adversarial attacks on neural networks.

Contribution

It introduces a novel coded search direction framework for distributed derivative-free optimization, extending evolution strategies with error correction capabilities.

Findings

Significant reduction in computation times for black-box adversarial attacks

Enhanced robustness to stragglers in distributed optimization

Effective application to deep neural network attacks

Abstract

We introduce a novel distributed derivative-free optimization framework that is resilient to stragglers. The proposed method employs coded search directions at which the objective function is evaluated, and a decoding step to find the next iterate. Our framework can be seen as an extension of evolution strategies and structured exploration methods where structured search directions were utilized. As an application, we consider black-box adversarial attacks on deep convolutional neural networks. Our numerical experiments demonstrate a significant improvement in the computation times.

Figures

Equations

$$\underset{\theta }{\mathrm{minimize}}$$

$$\frac{\partial f(x)}{\partial x_i}\approx \frac{f(x+\delta e_i)-f(x-\delta e_i)}{2\delta },$$

$$\nabla f(\theta )\approx \frac{1}{2N\delta }\sum _{i=1}^N(f(\theta +\delta {ϵ}_i){ϵ}_i-f(\theta -\delta {ϵ}_i){ϵ}_i)$$

$$\frac{\partial f}{\partial {\theta }_i}=e_i^T\nabla f.$$

$${\nabla }_{v}f=\underset{\delta \to 0}{\lim }\frac{f(\theta +\delta v)-f(\theta )}{\delta }.$$

$${\nabla }_{v}f=v^T\nabla f.$$

$${\nabla }_{v}f\approx \frac{f(\theta +\delta v)-f(\theta )}{\delta }.$$

$${\nabla }_{v}f\approx \frac{f(\theta +\delta v)-f(\theta -\delta v)}{2\delta }.$$

$$f(\theta +\delta v)$$

$$f(\theta -\delta v)$$

$${\nabla }_{v}f$$

$$=\nabla f^{T}v+\mathcal{O}({\delta }^2).$$

$$f(\theta )=||\theta -{\theta }_0|{|}_2^2+c\max \{\underset{i\ne t}{\max }[F(\theta ){]}_i-\log [F(\theta ){]}_t,-\kappa \}.$$

$$f(\theta )$$

$$||\theta -{\theta }_0|{|}_2^2+c\max \{\log [F(\theta ){]}_{t_0}-\underset{i\ne t_0}{\max }\log [F(\theta ){]}_i,-\kappa \}$$

Full text

I Introduction

Derivative-free optimization is an important computational task in many areas such as machine learning, statistics, design optimization and decision making [6]. In many applications, it is common to encounter black-box optimization problems where the gradient of the objective function is not available. It is also possible that the gradient information is unreliable, or impractical to obtain. Derivative-free optimization methods employ only the function values to make progress toward an optimal solution. They can be used to directly tackle complex and diverse problems without an analytical form of the objective function. Additionally, since the function values can be evaluated independently in parallel, derivative-free methods can exploit parallelism to a great extent.

In the recent years, cloud computing has offered inexpensive and scalable solutions to computational problems, and is widely adopted and used by the community. However, cloud-based serverless computing systems typically suffer from system noise, latency and variability in computation times [7, 8]. This results in a subset of slower worker nodes called stragglers. In this work, we propose a distributed derivative-free optimization method that is resilient to stragglers. Our method forms gradient estimates by evaluating the function at several points by employing coding theoretic ideas and tools. We first encode ordinary basis perturbations using Hadamard transform into perturbation directions along the rows of the Hadamard matrix $H$. The objective function is evaluated at the encoded points in parallel, and then a gradient direction is determined from the available workers via decoding. We establish a connection between derivative free optimization via the well-known finite differences method, recently proposed structured evolution strategies [5], and polar codes [1]. The structured evolution strategies approach corresponds to averaging the outputs of function evaluations with variables perturbed along the rows of $HD$ where $H$ is the Hadamard matrix and $D$ is a diagonal matrix with random diagonal entries. When the decoding step is omitted, encoding the unit perturbations leads to the structured evolution strategies which is simple to implement. If we decode the outputs instead of averaging them using the modified successive cancellation decoder of polar codes, however, we obtain partial derivatives as in the finite differences method. Because the function evaluations are scalar valued and the decoder can run in near linear time, the time it takes to decode these scalar outputs is negligible. For a large class of problems, including the decoding step yields faster convergence in optimization. The proposed method also enables a way to switch between finite differences and structured evolution strategies and provide a more favorable trade-off.

I-A Related Work

This work draws ideas from coded computation literature where the goal is to speed up distributed matrix multiplication. The authors in [11] introduced the concept of using Maximum Distance Separable (MDS) codes to compute matrix-vector multiplications faster in distributed computing platforms. The way coding is integrated into the computation of a matrix vector product $Ax$, where $A\in\mathbb{R}^{n\times d}$ and $x\in\mathbb{R}^{d}$ is as follows: First $A$ is partitioned into submatrices along the rows and then these submatrices are encoded using MDS codes. The workers multiply their assigned submatrix with $x$. Because the encoded $A$ is redundant, it is possible to decode $Ax$ via a standard decoder without accessing the outputs of all workers. Works that introduce variants and extensions of this idea include [14], [12] and [2]. There are many other recent studies on coded computation that explore the partitioning of both matrices being multiplied, e.g., [16], [17]. This work is more related to the case where only one of the matrices is partitioned.

Another line of study that this work is connected to is the study of black-box optimization methods for reinforcement learning. The authors in [13] explore the use of the evolution strategies method in reinforcement learning and show that distributed training with evolution strategies can be very fast because of its scalability. The work in [5] shows that using orthogonal exploration directions leads to lower errors and present the structured evolution strategies method which is based on a special way of generating random orthogonal exploration directions, as we discuss later in detail.

I-B Main Contributions

We introduced a novel framework based on coded gradient estimates that leads to straggler-resilient distributed optimization of black-box functions. Our method introduces a novel connection between the finite differences method and the structured evolution strategies method by using polar codes [1]. Once function evaluations at perturbed parameters are returned, it is possible to use them as they are without decoding, which is equivalent to the structured evolution strategies method or decode them to obtain the gradient estimate and use it in a gradient-based optimization algorithm. Our method offers this flexibility at the expense of doing some additional inexpensive computing for decoding because of the efficient polar code decoder.

II Preliminaries

II-A Notation

We are interested in solving the black-box optimization problem given by

[TABLE]

where $f:\mathbb{R}^{d}\to\mathbb{R}$. Throughout the text, unless otherwise stated, we assume that we don’t have access to the analytical form of $f(\theta)$ or its gradient, and we assume that we are only able to make queries for function evaluations. We use $\nabla_{v}$ to denote directional derivative along the direction $v$. We use $e_{i}$ to denote the $i$’th unit column vector with the appropriate dimension. Throughout the text, $H$ refers to the Hadamard matrix (its dimension can be determined from the context), and $\mathbf{H}$ refers to the Hessian of a function.

II-B Finite Differences

For black-box optimization problems, approximate gradients can be obtained by using the finite differences estimator. The derivative of a function $f(x)$ with respect to the variable $x_{i}$ can be approximated by using

[TABLE]

where $e_{i}$ is the $i$’th unit vector and $\delta\in\mathbb{R}$ is a small scalar that determines the perturbation amount. One can obtain an approximate gradient using (1) and feed these estimate gradients to gradient-based optimization methods such as gradient descent.

The finite differences estimator can easily exploit parallelism since the partial approximate derivatives can be computed in different workers in a distributed setting. This method can be modified to be straggler-resilient by using only the available derivative estimates and ignoring the outputs of the slower workers. However, this typically leads to slower convergence. We note that the proposed method results in a mechanism that always returns all entries of the gradient estimate and is straggler-resilient at the same time. Another alternative is to replicate finite difference calculations, which is not optimal from a coding theory perspective.

II-C Evolution Strategies

Let us consider the following evolution strategies (ES) gradient estimator ([13], [5])

[TABLE]

where $\delta$ is the scaling coefficient for the random perturbation directions $\epsilon_{i}$ and $N$ is the number of perturbations. This estimator is referred to as antithetic evolution strategies gradient estimator. There are other gradient estimators slightly different from the one in (2). In this work, we always consider the antithetic version given in (2) because it leads to better empirical results in our numerical simulations. The random perturbation directions $\epsilon_{i}$ may be sampled from a standard multivariate Gaussian distribution $\mathcal{N}(0,I)$. Alternatively, $\epsilon_{i}$’s may be generated as the rows of $HD$ where the $D$ is a diagonal matrix with entries distributed according to Rademacher distribution. The authors in [5] propose generating perturbation directions by repetitive multiplications $HD_{1}...HD_{k}$ where $D_{i}$’s are independent.

It is shown in [5] that if exploration (or perturbation) directions $\epsilon_{i}$ are orthogonal, the gradient estimators lead to lower error. We note that the rows of $HD$ are orthogonal with the appropriate scaling factor. We omit this scaling factor by absorbing it in the $\delta$ term which scales perturbation directions $\epsilon_{i}$. This work considers the case where random perturbation directions are generated according to $HD$ and we will refer to it as structured evolution strategies as they do in [5].

Evolution strategies can also exploit parallelism as well since workers need to communicate only scalars which are the function evaluations and the random seeds used when generating the random perturbation directions.

II-D Coded Computation

Coded computation has recently been introduced in [11] for speeding up distributed matrix multiplication. The idea is to add redundancy to computations by using linear codes in order to make it possible to recover the output of the multiplication without having all worker outputs. The fact that matrix multiplication is a linear operation plays a key role in coded computation. The scheme in [11] is based on MDS codes. More recently, the authors in [2] proposed a coded computation mechanism for serverless computing platforms using polar codes instead of MDS codes. Polar codes have low encoding and decoding complexity $O(N\log N)$.

This work draws ideas from the coded computation literature. In coded computation, for computing $Ax$, one encodes submatrices $A_{i}$ and decodes worker outputs $A_{i}x$. In coded optimization, the perturbation directions are encoded and function evaluations are decoded.

Many of the coded computation schemes have the restriction that they are designed to work with finite field data and not applicable to full-precision inputs. The decoding algorithm introduced in [2] for polar codes can however work with full-precision data. Since we do not restrict the output of the black-box function $f$ to take values in a finite field, we need a decoder that can operate on full-precision real values. Quantization is an option, though it would introduce noise, which can be avoided by using a decoding algorithm that can work with full-precision data. We note that it is possible to use the decoder in [2] with a slight modification to account for the difference in the encoding kernels (\bigl{[}\begin{smallmatrix}1&1\\ 0&1\end{smallmatrix}\bigr{]} for polar codes, and \bigl{[}\begin{smallmatrix}1&~{}\,\,1\\ 1&-1\end{smallmatrix}\bigr{]} for the Hadamard transform). We discuss the decoding in detail in the next section.

III Coded Black-Box Optimization

In this section, we present the proposed method for speeding up distributed black-box optimization. We start by introducing some more notation and definitions that we will be using. The derivative of a differentiable function $f$ at a point $\theta$ along the unit vector direction $e_{i}$ is the $i$’th component of the gradient $\nabla f$. More precisely, we have

[TABLE]

The directional derivative along $v$ is defined as follows

[TABLE]

If the function is differentiable at a point $\theta$, then the directional derivative exists along any direction $v$ and is a linear map [3]. In this case we have

[TABLE]

When we do not have access to exact gradients, we can employ a numerical directional derivative by choosing a small $\delta$ in

[TABLE]

Note that the approximation in (6) is not symmetric, that is, we only perturb the parameters along $+v$. We instead use the symmetric version of (6) for approximating derivatives in which the parameters are perturbed along both the directions $-v$ and $+v$:

[TABLE]

If we consider the Taylor series expansion for $f(\theta+\delta v)$, we obtain

[TABLE]

and for $f(\theta-\delta v)$, we have

[TABLE]

where $\mathcal{O}(\delta^{3})$ is a third order error term and $\mathbf{H}$ is the Hessian matrix for $f$. Substituting these expansions in (7), we obtain

[TABLE]

This shows that by choosing $\delta$ small, we can make the numerical directional derivative approximately linear in $\nabla f$. Our proposed method makes use of this assumption that gradient estimates are linear in their directions $v$ to ensure that coding can be applied to directional derivative estimates. To make this more concrete, let us consider the construction given in Figure 1. The block $H$ simply corresponds to the linear transformation H=\bigl{[}\begin{smallmatrix}1&1\\ 1&-1\end{smallmatrix}\bigr{]}.

In Figure 1, if we know the estimates for $\nabla_{(\theta_{1}+\theta_{2})}f$ and $\nabla_{(\theta_{1}-\theta_{2})}f$, we can compute the estimates for $\nabla_{\theta_{1}}f$ and $\nabla_{\theta_{2}}f$ because the directional derivative estimates are approximately linear in their corresponding directions. Furthermore, if we know the estimate for $\nabla_{\theta_{1}}f$, then it is sufficient to know only one of the estimates for $\nabla_{(\theta_{1}+\theta_{2})}f$ or $\nabla_{(\theta_{1}-\theta_{2})}f$ in order to be able to compute the estimate for $\nabla_{\theta_{2}}f$. This would happen if, for example, $\theta_{1}$ is the zero vector (i.e. frozen direction) because the estimate for $\nabla_{\theta_{1}}f$ would be zero and either of the directional derivative estimates from the right-hand side would be enough for us to obtain the estimate of $\nabla_{\theta_{2}}f$.

We refer to the directions $\theta_{1}$, $\theta_{2}$, $(\theta_{1}+\theta_{2})$, $(\theta_{1}-\theta_{2})$ as perturbation directions.

We now summarize the proposed method and detailed description of each item will follow:

• •

Encode all the unit vectors in $\mathbb{R}^{d}$ to obtain the encoded perturbation directions.

• •

Assign each perturbation direction to a worker and have workers compute their directional derivative estimates using (7).

• •

Master starts collecting worker outputs.

• •

When a decodable set of worker outputs (i.e. directional derivative estimates) is available, the master node decodes these outputs to obtain an estimate for the gradient.

• •

The master node then makes a gradient update to find the next iterate for the parameter $\theta$.

• •

Repeat until convergence or for a desired number of iterations.

The above procedure assumes that we want to estimate all entries of the gradient, but it is possible estimate only a portion of gradient entries by encoding only the unit vectors corresponding to the desired entries.

We note that we can always check whether decoding helps in obtaining a better objective function compared to the the structured evolution strategies and make the update accordingly. Because the decoding step is quite fast thanks to the polar decoder, having decoded the outputs but ended up not using the decoded estimate is not a computational burden.

III-A Encoding

Encoding is done based on the Hadamard transformation whose kernel is shown in Figure 2. Encoding is done the same way as polar encoding is done. Hence, the computational complexity of encoding is only $N\log N$ operations.

The way polar coding for the erasure channel works is that one computes the erasure probabilities of the transformed channels and freezes the channels with the highest erasure probabilities and the remaining channels are used for information. We refer the reader to [1] and [2] for more details on polar codes. In channel coding, freezing channels corresponds to sending known bits, e.g., zeros, at the receiver. In this work, freezing channels corresponds to setting it to zero coordinates such that the directional derivative along these vectors is zero. This makes it possible, when decoding, to take the value of the derivative estimates for frozen channels to be zero. For the information channels, we simply send in unit vectors.

Encoding for a function of $3$ variables using $N=4$ workers is shown in Figure 3. The resulting $4$ output vectors are the perturbation directions.

In this construction, the rate is $\frac{3}{4}$ since $3$ out of $4$ channels are used for sending in unit vectors. When the erasure probabilities of the transformed channels are computed (see [1]), one will see that the worst one corresponds to the first index. Hence, the first channel is the frozen channel and the remaining $3$ channels are the information channels. The frozen channel is set to the zero vector and the information channels are set to unit vectors. Sending in the zero vector for the frozen channel makes sense because perturbing by the zero vector is the same as not perturbing the variables and hence we get $f(\theta+0)-f(\theta-0)=0$. This is important because during decoding, we will not have to do any computations to evaluate the value of the frozen inputs as we know they are zero.

If we wish to use $N=8$ workers instead of $4$, we would set the worst $5$ channels to zero vectors and the remaining best three channels would be set to the unit vectors $e_{1},e_{2},e_{3}$. The rate in this case would be $\frac{3}{8}$, and this construction would be more straggler-resilient since we can recover the gradient estimate in the presence of even more stragglers compared to the $N=4$ case.

Embedding interpretation: It is also possible to perform the encoding step slightly differently for a different view on freezing channels. Instead of setting frozen channels to zero vectors, one can increase the dimension of the inputs from $d$ to $N$ and set the frozen channels to unit vectors $e_{j}$ where $j\in\{d+1,d+2,...,N\}$. Because the function $f(\theta)$ accepts $d$-dimensional inputs, we could embed $f(\theta)$ into a higher dimension, that is, we could define $\tilde{f}(\tilde{\theta})$ where $\tilde{\theta}\in\mathbb{R}^{N}$ and $\tilde{f}(\tilde{\theta})=f(\theta)$ if $\tilde{\theta}_{i}=\theta_{i}$ for $i\in\{1,2,...,d\}$. The advantage of this approach is that the output of the encoding step will be equal to the $H$ matrix with permuted rows.

III-B Decoding

The sequential polar code decoder given in [2] works for linear operations such as matrix-vector multiplication with full-precision data (i.e. does not require finite field data). Because we assume that the gradient estimates $\nabla_{h_{i}}f$ are approximately linear, we use the same decoder in [2] by only modifying it slightly so that it decodes according to the Hadamard kernel and not the polar code kernel. The decoder is given in Alg. 1 along with the subroutine decodeRecursive(i,j) in Alg. 2. The decoder uses the notation $D_{ij}$ to denote the value corresponding to the node $i$ in the $j$’th level in the coding circuit. Figure 4 shows this notation in the coding circuit for $N=4$. The blocks $W$ in Figure 4 represent workers. The notation $I_{D_{ij}}$ in Alg. 1 indicates whether the data for the node $i$ in the $j$’th level is available or erased (i.e., not computed yet). checkDecodability is another subroutine that checks whether it is possible to decode the available outputs given the indicators of output availabilities. It checks decodability of outputs by running the decoder on the indicators and if decoding fails, it returns false, otherwise, it returns true.

III-C Multiplication by $D$

Structured evolution strategies method is usually based on exploring the parameter space along the rows of $HD$ instead of $H$ alone. $D$ is a diagonal matrix with random diagonal entries distributed according to Rademacher distribution ($\pm 1$ with probability $0.5$). So far, we have only considered the $H$ matrix for perturbation directions. It is possible to incorporate the diagonal matrix $D$ into our method as well.

Multiplying $H$ by $D$ from the right corresponds to multiplying all the entries of the $i$’th column of $H$ by $D_{ii}$ for all $i$. In this case, instead of computing an estimate for the directional derivative along a direction of $v$, we will approximate the directional derivative with respect to the direction $Dv$. The derivations we have done for $v$ also apply for these randomized directions $Dv$. Decoding will work the same as before, except, after the decoding algorithm is run on the outputs, we need to divide the $i$’th entry of the decoded gradient estimate by $D_{ii}$ to obtain the entries of the gradient estimate.

IV Speeding Up Distributed Computation of Adversarial Attacks

Deep neural networks have found success in many different tasks. Their robustness is a critical performance criterion for many applications. In the literature, there are many works that propose methods to generate adversarial inputs to attack neural networks to investigate their robustness. Attack settings may differ in terms of levels of knowledge on the neural network model. Some works assume knowledge on the architecture of the neural network and this setting is referred to as open-box setting. On the other hand, if we only assume we have access to the outputs of the neural network, then this is called the black-box setting. In this section, we are interested in applying our proposed method to the black-box setting.

We apply our proposed method to the setting of finding adversarial examples based on black-box optimization. We adopt the settings in [4] where they develop black-box attacks to neural network based classifiers and the goal is to find a misclassified image that looks like a given image $\theta_{0}$. They pose the following optimization problem for finding adversarial examples: minimize $f(\theta)$, where

[TABLE]

$F(\theta)$ represents the class probabilities of the neural network when we feed in $\theta$ as the input. The subscript $i$ in $[F(\theta)]_{i}$ means the class probability for the $i$’th class. $c$ and $\kappa$ are non-negative parameters for tuning the attack. Note that the objective in (11) is for a targeted attack, meaning we wish to find an image that is classified as class $t$. If one is interested in finding an example misclassified but not necessarily to a given class, then this is called an untargeted attack. The optimization problem to be solved for untargeted attacks is given in [4] as

[TABLE]

where $t_{0}$ is the correct class for the image that we wish to misclassify. Note that since we assume we have no information about the neural network $F(\theta)$, we cannot compute the gradient of the functions in (11) and (IV).

The proposed approach in [4] involves selecting a given number of parameters in every iteration and estimating the derivatives only for these parameters. Hence, in every iteration, only a number of selected parameters are updated. The updates are done using the ADAM algorithm [9]. We present results regarding the use of our method in black-box adversarial attacks problem in numerical results section.

V Numerical Results

We now present results showing the performance of our proposed framework.

V-A An $l_{1}$-norm Based Objective as the Black-Box Function

In this subsection, we present experiment results for optimizing a known objective function $f(\theta)=||A\theta-b||_{1}$ as a test case. Even though we experiment with a known function, we still use the black-box optimization algorithms assuming we do not have the exact gradients.

Figure 5 shows the cost as a function of iterations when we use gradient descent algorithm with gradient estimates obtained by the finite differences method, the proposed method, and the structured evolution strategies method. For fairness in comparing these methods, we used straggler-resilient versions of the finite differences method and the structured evolution strategies method in obtaining these results. To clarify, for the finite differences method, we wait for the first arriving $16$ worker outputs out of the total $32$ outputs to make a gradient update. For the structured evolution strategies method, the perturbation directions come from the rows of $HD$ matrix which is a $32\times 32$ matrix, so we wait for the first arriving $16$ outputs out of the $32$. Finally, the way we implement the proposed method is that we use a rate of $\frac{1}{2}$ with a total of $64$ workers and wait for the first decodable set of outputs out of a total of $64$ outputs. Figure 5 illustrates that having all the entries of the gradient estimate through decoding leads to faster convergence than having only a half of the entries of the gradient estimate. It also shows that the proposed method results in faster convergence compared to the structured evolution strategies method.

V-B Black-Box Adversarial Attacks

We now present some experiment results for our proposed method applied to finding adversarial examples. We have tested our proposed method for solving the optimization problem for targeted black-box attacks (objective of which is given in (11)) to a deep learning model. The model we considered is the VGG-16 architecture [15] trained on CIFAR-10 dataset [10] for image classification. VGG-16 architecture is a 16-layer convolutional neural network. CIFAR-10 is an image dataset with a total of 60000 images of size $32\times 32\times 3$ (50000 training and 10000 test images) with 6000 images per class for a total of 10 classes.

Before we delve into the details of the experiments, we show two original input images from CIFAR-10 and the generated adversarial images that look almost the same as the original images but are misclassified. Figure 6 shows the original images $\theta_{0}$ which are correctly classified by the neural network $F(\theta)$. Figure 7 shows the generated adversarial images obtained by solving the black-box optimization problem for targeted attacks where the target class is the dog class. The adversarial images in Figure 7 are both misclassified as dog. These adversarial images were obtained by using our proposed method.

We now discuss details of how we obtained these adversarial images. For obtaining these results, we have solved the problem of minimizing $f(\theta)$ where $f(\theta)$ is given in (11). We have set $\kappa=0$, $c=0.1$. We have solved this optimization problem using three different methods and compared them in Figure 9 via a run time simulation. The simulation was done by sampling worker run times from the distributions shown in Figure 8. Figure 8(a) shows the empirical histogram of worker times which were obtained for the AWS Lambda platform, which is a serverless computing platform. Figure 8(b) shows the histogram of worker times generated from the shifted exponential distribution.

Figure 9 illustrates that it is possible to eliminate the straggling workers effect by using the proposed method. The way finite differences and structured evolution strategies methods were implemented is not straggler-resilient in these simulations, that is, all worker outputs are waited for in every gradient update. We note that this is different from the results on the optimization of the $l_{1}$-norm based function given in previous subsection because we used straggler-resilient versions of the methods in those experiments. The point of the simulation results given in Figure 7 is not necessarily to compare the run times of each method but instead to showcase that our proposed method works for the recent problem of adversarial attacks to deep neural networks.

VI Conclusion

We have introduced a novel distributed black-box optimization mechanism based on coded perturbation directions. Because of the redundancy inserted during encoding, the proposed method is resilient against straggling workers in distributed computing platforms. This makes our method suitable for large scale serverless cloud-based systems where nodes are unreliable. Our work utilizes coding theoretic tools and ideas from coded distributed computation methods. Furthermore, the proposed method provides a connection between two black-box optimization methods: finite differences and structured evolution strategies, in a way that it is possible to go from one to the other and choose the one that drives the cost down more effectively. We have provided an argument why coded gradient estimates would work for a variety of cost functions and shown this through numerical examples.