Metaheuristic Design of Feedforward Neural Networks: A Review of Two Decades of Research

TL;DR

This review paper summarizes two decades of research on optimizing feedforward neural networks, highlighting traditional and metaheuristic methods, and discusses emerging directions like deep learning and quantum neural networks.

Contribution

It provides a comprehensive overview of FNN optimization techniques, connecting various research trends and identifying future challenges in the field.

Findings

Metaheuristic algorithms are increasingly used for FNN optimization.

Traditional gradient-based methods have limitations leading to exploration of alternative approaches.

Emerging research directions include deep learning and quantum neural networks.

Abstract

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

Figures

Equations

$$y_i={\phi }_i\left(\sum _{j=1}^{n^i}{w}_j^i{z}_j^i+b^i\right),$$

$${\text{w}}^{t+1}={\text{w}}^t+\Delta {\text{w}}^t,$$

$$\Delta {\text{w}}_i^t={\eta }^t{e}_i^t{\text{x}}_i^t,$$

$$c_f({\text{y}}_i,{\hat{\text{y}}}_i)=\frac{1}{N}\sum _{i=1}^N\sum _{j=1}^q{\left(y_{ij}-{\hat{y}}_{ij}\right)}^2,$$

$$g^t=\frac{\partial c_f}{\partial {\text{w}}^t},$$

$${\text{w}}^{t+1}={\text{w}}^t+\Delta {\text{w}}^t,$$

$$H^t={\nabla }^2{c}_f=\frac{{\partial }^2{c}_f}{\partial \text{w}},$$

$$\Delta {\text{w}}^t=-H^{{}^t-1}{g}^t,$$

$$\Delta {\text{w}}_l^t={\alpha }^t{\text{w}}_l^{t-1}+{\eta }^t{g}^t{\text{y}}_{l-1},$$

$$\Delta {\text{w}}^t=\sum _{k=1}^n\sum _{i_1,i_2,\dots ,i_k=1}^n\left({\theta }_{i_1,i_2,\dots ,i_k}\prod _{j=1}^k{w}_{ij}^{t-1}\right)$$

$$c_{gen}=c_{trn}+\lambda c_{add},$$

$$c_{reg}=\frac{1}{2}\sum _i^n{w}_i^2=\frac{1}{2}\parallel \text{w}{\parallel }^2.$$

$$c_{net}=\sum _i^z\sum _j^z{c}_{ij},$$

$$c_{net}=\alpha c_{trn}+\beta c_{cv}+\gamma c_{hid}+\delta c_{node}+\theta c_{fun},$$

Full text

Keywords: Feedforward neural network; metaheuristics; nature-inspired algorithms; multiobjective; ensemble.

1 Introduction

Back in 1943 McCulloch and Pitts [1] proposed a computational model inspired by the human brain, which initiated the research on artificial neural network (ANN). ANNs are capable of learning and recognizing and can solve a broad range of complex problems. Feedforward neural networks (FNNs) are the special type of ANN models. The structural representation of an FNN makes it appealing because it allows perceiving a computational model (a function) in a structural/network form. Moreover, it is the structure of an FNN that makes it a universal function approximator, which has the capabilities of approximating any continuous function [2]. Therefore, a wide range of problems is solved by the FNNs, such as pattern recognition [3], clustering and classification [4], function approximation [5], control [6], bioinformatics [7], signal processing [8], speech processing [9], etc.

The structure of an FNN consists of several neurons (processing units) arranged in layer-by-layer basis and the neurons in a layer have connections (weights) from the neurons at its previous layer. Fundamentally, an FNN optimization/learning/training is met by searching an appropriate network structure (a function) and the weights (the parameters of the function) [10]. Finding a suitable network structure includes the determination of the appropriate neurons (i.e., activation functions), the number of neurons, and the arrangements of neurons, etc. Similarly, finding the weights indicates the optimization of a vector representing the weights of an FNN. Therefore, learning is an essential and distinguished aspect of the FNNs.

Numerous algorithms, techniques, and procedures were proposed in the past for the FNNs optimization. Earlier, in FNN research, only the gradient-based optimization techniques were the popular choices. However, gradually because of the limitations of gradient-based algorithms, the necessity of metaheuristic-based optimization methods were recognized.

Metaheuristics formulate the FNN components, such as weights, structure, nodes, etc., into an optimization problem. Metaheuristics implement various heuristics for finding a near-optimum solution. Additionally, a multiobjective metaheuristic deals with the multiple objectives simultaneously. The existence of multiple objectives in the FNNs optimization is evident since the minimization of FNN’s approximation error is desirable at one hand, and the generalization and model’s simplification is at the other.

In a metaheuristic or multiobjective metaheuristic treatment to an FNN, an initial population of FNNs is guided towards a final population, where usually the best FNN is selected. However, selecting only the best FNN from a population may not always produce a general solution. Therefore, to achieve a general solution without any significant additional cost, an ensemble of many candidates chosen from a metaheuristic final population is recommended.

This article provides a comprehensive literature review to address the various aspects of the FNN optimization, such as:

The importance of an FNN as a function approximator and its preliminary concepts (Section 2), including the introduction to the factors influencing FNN optimization (Section 2.2) and introduction to the conventional optimization algorithms (Section 2.3). 2. 2.

The role of metaheuristics and hybrid metaheuristics in FNNs optimization (Section 3). 3. 3.

The role of multiobjective metaheuristics (Section 4) and the ensemble methods (Section 5). 4. 4.

The current challenges and future research directions (Section 6).

2 Feedforward neural networks

The intelligence of human brain is due to its massively parallel neurons network system. In other words, the architecture of the brain. Similarly, a proper design of an ANN offers a significant improvement to a learning system. The components, such as nodes, weights, and layers are responsible for the developments of various ANN models.

A single layer perceptron (SLP) consists of an input and an output layer, and it is the simplest form of ANN model [11, 12]. However, SLPs are incapable of solving nonlinearly separable patterns [13]. Hence, a multilayer perceptron (MLP) was proposed, which addressed the limitations of SLPs by including one or more hidden layers in between an input and an output layer [14]. Initially, the backpropagation (BP) algorithm was used for the MLP training [15]. A trained MLP is then found capable of solving nonlinearly separable patterns [15]. In fact, MLPs (in general FNNs) are capable of addressing a large class of problem pertaining to pattern recognition and prediction. Moreover, an FNN is considered as a universal approximator [2]. Cybenko [16] referring to Kolmogorov’s theorem111Kolmogorov’s theorem: “All continuous functions of $n$ variables have an exact representation in terms of finite superpositions and compositions of a small number of functions of one variable [17].” showed that an FNN with only a single internal hidden layer—containing a finite number of neurons with any continuous sigmoidal nonlinear activation function—can approximate any continuous function. Also, the FNN structure (architecture) is itself capable enough to be a universal approximator [18, 2]. Hence, several researchers praised FNN for its universal approximation ability [19, 20, 21, 22].

Many other ANN models, like radial basis function [23] and support vector machine [24] are a special class of three-layer FNNs. They are capable of solving regression and classification problems using supervised learning methods. In contrast, adaptive resonance theory [25], Kohenen’s self-organizing map [26], and learning-vector-quantization [26] are two-layer FNNs that are capable of solving pattern recognition and data compression problems using unsupervised learning methods.

Additionally, the ANN architecture with feedback connections, in other words, a network where connections between the nodes may form cycles is known as a recurrent neural network (RNN) or feedback network model. The RNNs are good at performing sequence recognition/reproduction or temporal association/prediction tasks. RNNs such as Hopfield network [27] and Boltzmann machine [28] are good at the application for memory storage and remembering input–output relations. Moreover, Hopfield network was designed for solving nonlinear dynamic systems, where the stability of a dynamic system is studied under the neurodynamic paradigm [27].

A collection of RNN models, such as temporal RNN [29], echo state RNN [30], liquid state machine [31] and backpropagation de-correlation [32] forms a paradigm called reservoir computing, which addresses several engineering applications including nonlinear signal processing and control. Although some other ANN models that are capable of doing a similar task that of the FNNs were pointed out in this Section, the discussion in this article is; however, limited to only FNNs.

2.1 Components of FNNs

FNNs are the computational models that consist of many neurons (node), which are connected using synaptic links (weights) and are arranged in layer-by-layer basis. Thus, the FNNs have a specific structural configuration (architecture) in which the nodes at a layer have forward connections from the nodes at its previous layer (Fig. 1(a)).

A node of an FNN is capable of processing information coming through the connection weights (Fig. 1(b)). Mathematically, the output $y_{i}$ (excitation) of a node (node indicated as $i$) is computed as:

[TABLE]

where $n^{i}$ is the total incoming connections, $z^{i}$ is the input, $w^{i}$ is the weight, $b^{i}$ is the bias, and $\varphi_{i}(\cdot)$ is the activation function at the $i$-th node to limits the amplitude of the output the node into a certain range.

Fig. 1(a) is a structural representation of an FNN, i.e., a phenotype of a function $f(\text{\bf{x}},\text{\bf{w}})$, which is parameterized by a $p$-dimensional input vector $\text{\bf{x}}=\langle x_{1},x_{2},\ldots,x_{p}\rangle$ and an $n$-dimensional real-valued weight vector $\text{\bf{w}}=\langle w_{1},w_{2},\ldots,w_{n}\rangle$. The function $f(\text{\bf{x}},\text{\bf{w}})$ is a solution of a problem. Therefore, two tasks involved in solving a problem using an FNN are: to discover an appropriate function $f(\text{\bf{x}},\text{\bf{w}})$ (i.e., the architecture optimization) and to discover an appropriate weight vector w (i.e., the weights optimization) using some learning algorithm.

The architecture optimization indicates the search for the appropriate activation functions at the nodes, the number of nodes, number of layers, the arrangements of the nodes, etc. Therefore, several components of an FNN optimization are: the connection weights; the architecture (number of layers in a network, the number of nodes at the hidden layers, the arrangement of the connections between nodes); the nodes (activation functions at the nodes); the learning algorithms (algorithms training parameters); and the learning environment. However, traditionally, the only component that was optimized was the weights of the connections by keeping other components fixed to the initial choice.

2.2 Influencing factors in FNN optimization

2.2.1 Learning environments

An FNN is trained by supplying the training data ($X,Y$) of $N$ input–output pairs, i.e., $X=(\text{{x}}_{1},\text{{x}}_{2},\ldots,\text{{x}}_{N})$ and $Y=(\text{{y}}_{1},\text{{y}}_{2},\ldots,\text{{y}}_{N})$. Each input $\text{\bf{x}}_{i}=\langle x_{i1},x_{i2},\ldots,x_{ip}\rangle$ is a $p$-dimensional vector, and it has a corresponding $q$-dimensional desired output vector $\text{\bf{y}}_{i}=\langle y_{i1},y_{i2},\ldots,y_{iq}\rangle$. For the training data ($X,Y$), an FNN produces an output $\hat{Y}=(\hat{\text{\bf{y}}}_{1},\hat{\text{\bf{y}}}_{2},\ldots,\hat{\text{\bf{y}}}_{N})$, where a vector $\hat{\text{\bf{y}}}_{i}=\langle\hat{y}_{i1},\hat{y}_{i2},\ldots,\hat{y}_{iq}\rangle$ is a $q$-dimensional FNNs output, which is then compared with the desired output $\text{\bf{y}}_{i}$, for all $i=1$ to $N$ by using some error/distance/cost function. The minimization/reduction of the error/distance function, in an iterative manner, is referred as a supervised learning. One very commonly known supervised learning algorithm is Delta rule or Widrow-Hoff rule [33, 34] in which the $n$-dimensional weight vector w of an FNN is optimized as:

[TABLE]

where $\Delta\mathrm{w}^{t}$ is weight change (an additive term) at $t$-th iteration. The weight change $\Delta\mathrm{w}^{t}$ is computed as:

[TABLE]

where $\eta^{t}$ is a learning rate, which controls the magnitude of weight change at $t$-th iteration and $e^{t}_{i}$ is the error at $t$-th learning iteration corresponding to $i$-th training input $\text{{x}}^{t}_{i}$ presented to an FNN. The error $e^{t}_{i}$ at the $t$-th iteration may be computed as: $e^{t}_{i}=\sum_{j=1}^{q}(y^{t}_{ij}-\hat{y}^{t}_{ij})^{2}$, where $y^{t}_{ij}$ and $\hat{y}^{t}_{ij}$ are the desired output and FNN’s output at $t$-th iteration respectively.

Contrary to the supervised learning paradigm, there are two other learning forms for the spacial cases of FNNs: 1) the unsupervised learning—for the unlabeled training data [35], and 2) the reinforcement learning—for the training data with insufficient input–output relations [36]. The focus of this article is, however, on supervised learning paradigms only.

2.2.2 Error functions

A supervised learning, essentially, is the minimization of the difference/distance between the desired output $\text{\bf{y}}_{i}$ and the model’s output $\hat{\text{\bf{y}}}_{i}=f(\text{\bf{x}},\text{\bf{w}})$ by comparing the difference/distance using a cost function $c_{f}:Y\times\hat{Y}\longrightarrow\mathbb{R}_{\geq 0}$. For this propose, several cost function can be designed. For instance, in regression problems, mean squared error is one of the commonly used cost function, which is written as:

[TABLE]

where $y_{ij}$ are the desired response and $\hat{y}_{ij}$ are the FNN’s responses, and their differences were summed over $N$ data pairs. Some other functions like sum of squared error, root of mean square error, mean absolute error, correlation coefficient, etc., can be used for evaluating the FNN’s predictability [37].

The cost function (4) or any similar squared-error-based cost function is inconsistent for solving classification problems [38]. Instead, the percentage of good classification, which has consistent behavior, can be used [38]. However, the percentage of good classification is satisfactory until no preference was given to a particular class. Therefore, accuracy and miss-classification rate are used as the cost functions. A detailed list of the cost function for evaluating the classification problems is available in [39, 40, 41].

In this article, cost function mentioned for FNN optimization is discussed in a general sense, which can be thought as the equivalent to any other user-defined cost function. Another factor related to fitness of an FNN is to compare the cost functions of two or more FNN models [42, 43]. Some researchers also argue to statistically compare the predicted outputs of two or more FNN models to establish the differences in their performances [44].

2.2.3 Local minima problem

Let $c_{f}:S\longrightarrow\mathbb{R}_{\geq 0}$, where $S\subset\mathbb{R}^{n}$ is nonempty and compact (for detailed information about topological compactness, see [45]). Therefore, the following may be defined:

Definition 1**.**

A point $\text{\bf{w}}^{*}\in S$ is called global minima if $c_{f}(\text{\bf{w}}^{*})\leq c_{f}(\text{\bf{w}})\text{ for any }\text{\bf{w}}\in S$ holds.

Definition 2**.**

A point $\text{\bf{w}}^{*}\in S$ is called local minima if there exists $\epsilon>0$, and an $\epsilon$-neighborhood $B_{\epsilon}(\text{\bf{w}}^{*},\epsilon)$ around $\text{\bf{w}}^{*}$ such that $c_{f}(\text{\bf{w}}^{*})\leq c_{f}(\text{\bf{w}})\text{ for any }\text{\bf{w}}\in S\cap B_{\epsilon}(\text{\bf{w}}^{*},\epsilon)$ holds.

Learning algorithms when to using the cost function (4) or any similar function for FNN optimization has the tendency to fall in local minima [46]. Moreover, the geometrical structure (parameter space) of a three-layer perceptron may fall to local minima and plateaus during its optimization. It indicates that the critical point corresponding to global minima of a smaller FNN model (model with $h-1$ hidden units) can be a local or saddle point of a larger FNN model (model with $h$ hidden units) [47]. However, there are some ways to avoid or eliminate local minima in FNN optimization [48, 49]:

If the weights and training patterns are assigned randomly to a three-layer FNN that contains $h$ neurons at the hidden layer, then a gradient-descent algorithm can avoid trapping into local minima [50]. 2. 2)

If linearly-separable training data and pyramidal network structure are taken, then the error surface will be local minima free [51]. 3. 3)

If there are $N$ many noncoincident input patterns to learn and three-layered FNN with $N-1$ sigmoid hidden neurons and one dummy hidden neuron is used, then the corresponding error surface will be local minima free. 4. 4)

If the training algorithms can be improved as similar to as the global descent learning algorithm proposed by Cetin et al. [52] to replace gradient-descent algorithms, then it can avoid local minima.

These four methods depend on the number of hidden neurons, the number of training samples, the number of output neurons, and a condition that says the number of hidden neurons should not be less than the number of training samples. Moreover, it does not necessarily guarantee to converge to global minima and to set preconditions for the number of hidden neurons and linearly separable training patterns are unlikely conditions for the real-world problems [53].

2.2.4 Generalization

The generalization is a crucial aspect of an FNN optimization, where it is an ability to offer the general solutions rather than performing best for the particular cases. To achieve generalization, FNNs need to avoid both underfitting and overfitting during training, which is associated with high statistical bias and high statistical variance [54]. Therefore, one has to address trade-offs between bias and variance. Also, for a good generalization, the number of training pattern should be sufficiently larger than the total number of connections in FNN [55].

The standard methods to achieving generalization are determining an optimum number of free parameters (i.e., equivalent to find an optimum network architecture), early stopping of training algorithms, adding regularization term with the cost function [56, 57], and adding noise to the training data.

In early stopping, a dataset is divided into three sets: a training set, cross-validation set, and test set. The early stopping scheme suggests stopping of training at the point (epoch) from which onward the cost function value computed on cross-validation set starts to rise [46, 58, 59, 60]. Similarly, adding noise (jitters) into the training pattern improves FNN’s generalization ability and removing insignificant weights from a trained FNN improves its fault tolerance ability [61]. Moreover, generalization is related to sparsity and stability of a learning algorithm [62].

Now, if the approximation error of two FNN models trained on the same training data is close/similar, then the model with simple network structure (lower number of free parameters) should be selected as the best model. It is because the model with lower network complexity possesses higher generalization ability than the models with higher network complexity [63]. Moreover, the network with lower weight magnitude possesses better generalization ability [63].

2.3 Conventional optimization approaches

Finding a suitable algorithm for the FNNs optimization has always been a difficult task. The FNN optimization using conventional gradient based algorithms is viewed as an unconstrained optimization problem [10, 64]. The cost function $c_{f}$ has to be optimized to satisfy Definition 1. Therefore, the gradient of error $g^{t}$ at $t$-th iteration is computed as:

[TABLE]

where $g^{t}$ is a first-order partial derivative of the cost function $c_{f}$ with respect to weight vector w. Hence, a gradient-descent approach starts with an initial guess $\text{\bf{w}}_{0}$ and generates a sequence of weight vector $\text{\bf{w}}_{1},\text{\bf{w}}_{2},\ldots$ such that $c_{f}$ reduces in each iteration. The connection weights at iteration $t$ are updated as:

[TABLE]

where the weight change $\Delta\text{\bf{w}}^{t}\text{ is equal to }-\eta^{t}g^{t}$, and $\eta^{t}$ is the learning rate. The weights updated using (5) and (6) is known as the steepest decent approach. Now, instead of using a first-order partial derivative, a second-order partial derivative ($\nabla^{2}$) of cost function $c_{f}$ can be used as:

[TABLE]

where $H^{t}$ is Hessian matrix at the $t$-th iteration [65]. Hence, the weight change $\Delta\text{\bf{w}}^{t}$ using second–order Taylor’s series expansion of cost function $c_{f}$ around point $\text{\bf{w}}^{t}$ is computed as:

[TABLE]

where $H^{{}^{t}-1}$ is the inverse of Hessian matrix $H^{t}$ and the weight change $\Delta\text{\bf{w}}^{t}$ is known as the Newton method or Newton update [10]. In the past, several algorithms were proposed using (5) and (8). Some of them are summarized as follows:

Backpropagation (BP) is a first-order gradient-descent algorithm for the FNNs optimization [14, 15]. In BP, the error computed at the output layer is propagated backward to the hidden layers. BP algorithm has two phases of computation: forward computation and backward computation, where at $t$-th iteration, the weight change $\Delta\text{\bf{w}}^{t}$ for $l$-th layer is computed as:

[TABLE]

where y is inputs/excitation from previous layer $l-1$, $\eta^{t}$ is learning rate and $\alpha^{t}$ is momentum factor.

The choice of learning rate $\eta^{t}$ and momentum factor $\alpha^{t}$ are critical to gradient-descent technique. The momentum factor $\alpha^{t}$ allows BP training to be biased with previous iteration weights that help convergence rate to be faster. BP is sensitive to these parameters [15]. If the learning rate is too small, learning will become slow, and if the learning rate is too large, learning will be zigzag and algorithm may not converge to required degree of satisfaction. Additionally, a high momentum factor leads to a high risk of overshooting minima and a low momentum factor may avoid local minima, but learning will be slow. The classical BP algorithm is slow and has a tendency to fall in local minima [51].

Since the basic version of BP is sensitivity towards learning rate and momentum factor [15], several improvements were suggested by researchers: 1) a fast BP algorithm, called Quickpro was proposed in [66, 67]; 2) a delta-bar technique and an acceleration technique was suggested for tuning BP learning rate $\eta$ in [68] and in [69] respectively; and 3) a variant of BP, called resilient propagator (Rprop) was proposed in [70].

In the Rprop, if the gradient direction in iteration $n$ remains unchanged from its previous iteration $t-1$, then the weight change will occur in larger magnitude, else in smaller. In simple words, if gradient sign remains unchanged from previous iterations, the magnitude of learning rate $\eta$ will be large, otherwise small. The proposed Rprop improves determinism of convergence to global minima [70]. However, it is not faster than the Quickpro, but still faster than BP [71].

Contrary to BP, a second-order minimization method, called conjugate gradient (CG) can be used for weights optimization [72, 73, 74]. The CG does not proceed down with a gradient; instead, it moves in the direction that is conjugate to the direction of the previous step. In other words, the gradient corresponding to the current step stays perpendicular to the direction of all the previous steps, and each step is at least as good as its previous step. Such series of steps are non-interfering. Hence, the minimization performed in one step will not be undone by any further steps. Several variants of the CG were proposed in the past [75].

Similar to the CG, many other variants of derivative-based conventional methods are used for weights optimization: Quasi-Newton [65], Gauss-Newton [76], or Levenberg-Marquardt [77]. Quasi-Newton uses a second-order partial derivative (7) of error (4), and it computes its weight search direction by using Broyden-Fletcher-Goldfarb-Shanno (BFGS) method [78]. In Gauss-Newton method, the FNNs optimization is framed as a nonlinear least square optimization problem, which suggests to using the sum of squared error (4) [77]. Many researchers suggested that the Levenberg-Marquardt (LM) method outperforms BP, CG, and Quasi-Newton methods [79, 80]. Several other methods were proposed for the FNNs optimization are based on Kalman-filter [81, 82] and recursive least squares method [83].

2.4 Comments on conventional approaches

The gradient-descent based conventional algorithms operate on a single solution (a weight vector) during the optimization. Thus, these algorithms are computationally faster than the algorithms that use two or more solution vectors during the optimization and select the best solution vector at the end of optimization iterations. Moreover, the gradient-decent methods such as BP [15], Online BFGS [84] can be applied for the stochastic as well as batch mode training of the FNNs.

The basic advantages of the stochastic/online training of an FNN are its ability to address redundancy in training pattern, the inclusion of training data that are currently not in training set (i.e., the possibility of dynamic learning), and faster training than that of batch mode [85]. Whereas, most of the other second-order gradient-descent methods and metaheuristic algorithms can only use batch mode training. However, a batch mode (offline) training of an FNN can at least guarantee a local minima under a simple condition compared to a stochastic/online training, and for a larger dataset, batch mode training can be faster than stochastic training [86].

On the contrary to the advantages of the conventional methods, they have several limitations. For example, they have the tendency to fall in local minima, and they are only used for optimizing the FNN weights. Primarily, the gradient-descent algorithms depend on the error function, e.g., mean square error or sum of squared error. For instance, the least square methods like the Gauss-Newton and LM works only if the cost function is the sum of squared error. The Newton method has to compute a Hessian matrix (7), which has to be positive definite and computing the Hessian matrix (7) can be hard and expensive. Similarly, the Quasi-Newton and CG methods need to use a line-search method that sometimes can be expensive.

Moreover, the FNN generalization, as mentioned in Section 2.2.4, needs the reduction in the number of weights (less complex network architecture). Hence, the application of conventional algorithms is limited compared to the metaheuristic algorithms such as the genetic algorithm (GA) that can be directly applied to an FNN for its automatic structure determination and complexity reduction [87, 88]. Similarly, a metaheuristic algorithm can formulate an FNN such that the insignificant weights of the network can be eliminated to improve the FNN generalization ability. Moreover, metaheuristic algorithms can evolve an FNN as a whole by optimizing its components simultaneously.

3 Metaheuristic approaches

So far, only the gradient-descent based algorithms were discussed, which are local search algorithms. They are good at exploiting the obtained solutions to find new solutions. However, to find a global optimum solution, any optimization algorithm must use two techniques: 1) exploration—to search new and unknown areas in a search space, and 2) exploitation—to take advantage of the already discovered solution [89]. The exploration and exploitation are two contradictory strategies and a good search algorithm must find a trade-off between these two. Metaheuristic is the procedure that implements nature-inspired heuristics to combine these two strategies [90]. Hence, metaheuristic approaches are alternative to the conventional approaches for optimizing the FNNs.

Unlike the conventional methods, which require the objective function to be continuous and differential, the metaheuristic algorithms have the ability to address complex, nonlinear, and non-differentiable problems. However, the optimization algorithms are often biased towards a specific class of problems, that is, “there is no such universal optimizer which may solve all class of problem,” which is evident from no free lunch theorem [91].

Wolpert and Macready [91] introduced no free lunch (NFL) theorem to answer the question, “whether a general purpose optimization algorithm exists.” Moreover, Wolpert [92] introduced NFL for optimization algorithm to answer the question, “How does the set of problems $F_{1}\subset\mathcal{F}$ for which algorithm $a_{1}$ performs better than algorithm $a_{2}$ compares to the set $F_{2}\subset\mathcal{F}$ for which the reverse is true.” Here, $\mathcal{F}$ is space of all possible problems. To answer this question, Wolpert proposed NFL theorem, which says that “the average performance of any pair of algorithms across all possible problems is identical” [91].

Therefore, a straightforward interpretation of NFL is as follows. “A general purpose universal optimization strategy is impossible, and the only way one strategy can outperform another if it is specialized to the structure of the specific problem under consideration” [93]. Schumacher et al. [94] argue that the NFL theorem [91] holds only for the set of problems which are closed under permutation (c.u.p). Therefore, indeed the performance of one algorithm can be improved over another for the problems that are not c.u.p and most of the real-world problems are not c.u.p [95]. Such is the reason why in the past researchers were inclined to create and improvise algorithms for optimizing the FNNs.

3.1 Metaheuristic algorithms

Since a large variety of metaheuristic algorithms are available, it is difficult to classify metaheuristic algorithms precisely into different classes. Though, intuitively, three basic categorize can be done:

3.1.1 Single solution based algorithms

A single solution based metaheuristic algorithm operates on a single solution (candidate) and applies some heuristic inspired by the nature or some other phenomena on the current solution. For example, algorithms like simulated annealing (SA) [96], tabu search (TS) [97], variable neighborhood search (VNS) [98], greedy randomized adaptive search (GRAP) [99], etc., improves a single solution by searching around its neighborhood and continue to improve the solution until a satisfactory solution is obtained. For instance, the heuristics of some algorithms are as follows:

SA is a probabilistic approach that imitates the cooling strategy (annealing process) of a metallurgy industry. It uses Monte Carlo method [100] to determine the acceptance probability of a newly generated solution in the neighborhood of the current solution. Hence, for a given search space, SA should guide a solution towards a global optimal solution [96, 101]. Similarly, TS is inspired by the human behavior of tabooing objects [97]. In other words, TS discourages (tabu) the acceptance of the solutions that are already explored in the past. Hence, it improves upon SA by introducing some additional restriction on the acceptance of the new solutions.

Since a single solution based algorithm exploits the current solution, it also is known as the local search algorithm. The focus of this article is to illustrate the application of the metaheuristic algorithms for the FNNs optimization. Hence, for the detailed description of the mentioned algorithms, the readers may explore the respective references.

3.1.2 Population based algorithms

Population based algorithms operate on the multiple solutions (candidates) and apply the heuristics inspired by nature, biological evolution, biology, or some other forms. In contrast to the single solution based algorithms, the population based algorithms have a high exploration (global search) ability. The following are the population based algorithms:

Evolutionary algorithms (EA)

Genetic algorithms (GA) [87, 88], evolutionary programming (EP) [102], evolutionary strategy (ES) [103], genetic programming (GP) [104], differential evolution [105], etc., are the algorithms inspired by the dynamics of natural selection and use the operators, such as selection, crossover, and mutation to find a near-optimal solution [88]. EA framework offers an exploration of a vast search space and guarantees to find a near-optimal solution. Since EAs do not depend on gradient information, they solve a large range of complex, nonlinear, nondifferentiable, multimodal optimization problems. Also, EAs give a wider scope in the FNN optimization since EAs can optimize both discreet and continuous optimization problem, and the FNN components can be formulated into both ways.

The differences between EAs can be briefly stated as follows: GA uses genetic operators, such as selection, crossover, and mutation to search optimum genetic vector from a search space [88]; whereas, only the mutation operator are used in ES to evolve a real vector solution [103]. On the other hand, GP searches an optimum program structure from a topological search space of computer programs [104] and EP are used for evolving parameters of a computer program whose structure is kept fixed [102].

Swarm intelligence (SI)

SI algorithms are inspired by the collective and self-organized behavior of the swarm (insects, birds, fish, etc.). Particle swarm algorithm (PSO) [106], ant colony optimization (ACO) [107], artificial bee colony (ABC) [108], bacterial foraging optimization (BFO) [109], etc., are some widely used SI algorithms. The basic principle of SI algorithms is as follows. First, a swarm (collection of solutions) are randomly generated. Then, the heuristic inspired by the swarm behavior modifies the current solution. For example, in PSO, ACO, ABC, and BFO, the heuristics are inspired by the foraging behavior of birds, ants, bees, and bacteria respectively. In these algorithms, an FNN component is formulated as a solution for the optimization.

In PSO, a swarm, as a whole, is like a flock of birds (particles, which are the weight vectors) that collectively foraging (explore search space) for food (best weight vector) and is likely to move close to an optimum food-source [106, 110]. Moreover, each particle bears two properties: location and velocity. The location of a particle is a solution vector (weight vector w), and velocity $\eta$ is a vector equal to the size of the location vector. Each particle determines its movement using knowledge of its best locations, global best location, and random perturbations [106, 111].

In ACO, the artificial ants explore the area around their nest (colony) for searching a food source. ACO takes advantage of ants ability to choose the shortest path to a food source by communicating among each other’s using a pheromone secretion [112]. This behavior of ants led to the development of ACO algorithm [107].

Similarly, in ABC, three kinds of honey bees, such as employed bee, onlooker bee, and scout bee are responsible for searching food source [108]. Each employed bee memorizes a food source, i.e., a solution (weight vector). Then, each onlooker bee examines the nectar amount (fitness of solution) of a food source memorized by the employed bees and depending on nectar amount; they send scout bees for searching new food source. Hence, they iteratively construct the solution. The readers are encouraged to explore the detail description the algorithms in their respective references.

Other metaheuristics

The population based metaheuristic algorithms metaphor is exploited to device several algorithms. There are algorithms inspired by the behavior of animals, birds, and insects, such as gray wolf optimization (GWO) [113], flower pollination (FP) [114], cuckoo search (CS) [115], firefly (FF) [116], etc.

Similarly, there are algorithms inspired by some phenomenon observed in the physics and chemistry, such as harmony search (HS) [117], central force optimization (CFO) [118], gravitational search optimization (GSO)[119], etc. A detailed list and classification of metaheuristic algorithms are provided in [120].

The growing number of metaheuristic algorithms has drawn researchers to examine the metaphor, the novelty, and the significant differences among the metaheuristic algorithms [121, 122]. In [122], the author provided an insight of the metaheuristic developed over the time, starting from SA to TS, EA, ACO, HS, etc. The author claimed that most of the metaheuristic are similar in nature and do not provide a groundbreaking method in optimization. Despite the criticisms, the author acknowledged the quality of metaheuristic research has been produced and can be produced.

3.1.3 Hybrid and memetic algorithms

An effective combination of various metaheuristic algorithms may offer a better solution than that of a single algorithm. A paradigm of hybrid algorithms, called memetic algorithm gave a methodological concept for combining two or more metaheuristics (global and local) to explore a search space efficiently and to find a global optimal solution [123].

The conventional algorithms have the local minima problem because they lack global search ability, but they are fast and good in local search. On the other hand, the metaheuristics are good in global search, but they suffer premature convergence[124, 125]. Therefore, a combination of these two may offer a better solution in FNN optimization than that of using any one of them alone (Fig. 2). To reach a global optimum, a hybrid strategy can be used. Fig. 2 shows an impact of hybridization of metaheuristics on the FNNs optimization. The hybrid algorithms can be categorized in two paradigms:

The combination of conventional and metaheuristic algorithms—to take advantage of local search and global search algorithms. 2. 2)

The combination of two or more metaheuristic algorithms—to make use of different heuristics or a combined influence of two or more heuristics of global search algorithms.

Under the definition of memetic algorithms, researchers combine EAs with conventional algorithms [126]. For example, the effectiveness of global (GA) and local search (BP) combination is explained in [127, 128]. Similarly, a hybrid PSO and BP algorithms for optimizing FNN were found useful in obtaining better approximation than using one of them alone [129].

A convergence scenario similar to Fig. 2 was illustrated in [130], where ABC was applied for searching initial weights and LM was applied for optimizing the already discovered weights. An example of effectively combining two metaheuristic GA and PSO is illustrated in [131], where both GA and PSO optimized the same population. A detailed description of the hybrid metaheuristic algorithms for the FNN optimization is described in the following Section.

3.2 Metaheuristic formulation of the FNN components

Metaheuristics are stochastic/non-deterministic algorithms. Hence, they do not guarantee a global optimal solution, but they can offer a near-optimal (satisfactory) solution. Moreover, metaheuristics efficiently solve a wide range of complex continuous optimization problems; especially when the problems have incomplete and imprecise information.

The basic form of FNN optimization is the act of searching its weights (free parameters of FNN) such that the cost function (4) or similar function can be minimized. However, the goodness (performance) of FNN cost function depends not only on finding optimum weights, but finding the optimum architecture, activation function, parameter setting of learning algorithm, and training environment are equally important. To apply metaheuristic algorithms for optimizing an FNN, its components (phenotype) need to be formulated into a vector (genotype) form.

Usually, the FNN components, such as weights, architecture, activation function, learning rule, etc., are considered arbitrarily. Then, a learning algorithm is applied to search weights while the other components are kept fixed to their initial setting. The metaheuristics, on the other hand, allow us to optimization each component simultaneously or a combination of components efficiently (Fig. 3).

The Venn diagram in Fig. 3 illustrates the spectrum of FNN components optimization: weights, architecture, activation function, and learning rule’s parameters. In Fig. 3, area “a1” indicates the optimization of weights; area “a2” indicates the optimization of weights and architecture; area “a3” indicates the optimization of weights, architecture, and activation function; and all other possible combinations. Examining Fig. 3, one can say that the strength and complexity of optimization increases from area denoted “a1” to “a8,” where “a1” is the simplest approach and “a8” is the most sophisticated approach.

Each FNN component can be separately optimized on a one-by-one basis. Therefore, firstly, the weights can be optimized by keeping a fixed architecture; secondly, the architecture can be optimized keeping weights fixed; thirdly, the activation function can be optimized keeping architecture and/or weights fixed; and so on. Another way is to optimize all or a combination of FNN components simultaneously. Therefore, weights and architecture can be optimized, simultaneously; or weights and activation functions, simultaneously; or weights, architecture, and activation functions simultaneously; and so on. In the simultaneous optimization of all or a combination of components, a vectored representation of all components, or a combination of components can be optimized respectively. Once a vectored representation (genotype) is designed, then one of the available metaheuristic algorithms in the literature can be applied to optimize the designed vector to obtained an optimum FNN.

Now, there are single-solution based and population based metaheuristics [133]. In a single-solution based metaheuristic algorithm, a genotype $\text{\bf{w}}=\langle w_{1},w_{2},\ldots,w_{n}\rangle$ is used. Whereas, in a population-based metaheuristic algorithm, a collection of many genotypes are used. In other words, a population matrix $\text{\bf{W}}=(\text{\bf{w}}_{1},\text{\bf{w}}_{2},\ldots,\text{\bf{w}}_{m})$ of $m$ weight vectors is used.

Yao and Liu [134] identified evolution at various components of FNN that fall into the spectrum of metaheuristic design of FNN shown in Fig. 3. This Section will describe how researchers applied metaheuristics for evolving FNN. The evolution in FNN components is described here one-by-one, as follows. Here, the word optimization, adaptation, and evolution are used in the similar context.

3.2.1 Weight optimization

FNN weight optimization is the most common and widely studied approach, in which the weights are mapped onto an $n$-dimensional weight vector w, where $n$ is the total number of weights in a network. The vector w is a genotype representation of a phenotype (FNN structure), where the weight $\text{\bf{w}}\in\mathbb{R}^{n}$. The weights $w_{i}$, an element of vector w, may be encoded in the following ways: by assigning a real value, $l$-bits binary coding, $l$-bits gray coding, IEEE floating point coding, etc. Fig. 4 is an example of phenotype to genotype mapping, where a phenotype shown in Fig. 4 that has the connectivity matrix $c$ as per Fig. 4 is encoded into three different weight vectors shown in Fig. 4.

FNN weights optimization using metaheuristic is practiced from **early 80’s **when even the term metaheuristic was not used. Engle’s [135] work on FNN weight vector optimization using SA was the first evidence of metaheuristic application. To optimize weight vector using SA, first, the phenotype was mapped onto a real-valued weight vector (Fig. 4), and to compute fitness of the FNN, a reverse mapping from genotype (weight vector w) to phenotype (FNN) was used. Such process was continued until a satisfactory solution was found. SA based FNN weight optimization was found to be performing better in comparison to conventional approaches [136, 137, 138].

Similar to Engle’s [135] approach of phenotype to genotype mapping and vice versa, in [139], the FNN weights optimization was performed using TS. In [140], an improvised TS, called reactive tabu search was used for optimizing weights. Several studies show that TS when used for optimizing FNN weights, outperformed BP and SA algorithm [141, 142]. However, SA and TS are single solution based algorithms, which has a limited scope of exploring search space to obtain a global optimal solution. In contrast, the EAs, SI, or other bio-inspired metaheuristics are population-based algorithms that operate on multiple agents to explore a search space. Hence, they have a better exploration ability than SA, TS, BP, CG, and other single solution based algorithms [88, 110].

For optimizing the FNN weights, EAs use two types of vector representation: real-valued and binary valued vector representation. In Fig. 4, following weight vector representation are illustrated: 1) real-valued coded chromosome, 2) binary coded chromosome, and 3) binary gray-coded chromosome.

Goldberg and Holland [88] gave the idea of FNN training using GA. However, Whitley and Hanson [143] were the first to propose “GENITOR,” a GA based FNN training procedure that used binary-coded chromosome (Fig. 4) for optimizing the weights. Many others followed the idea of GENITOR with some additional improvements such as connectivity optimization and reduced search space introduction [144, 145]. On the other hand, in [146], a binary gray coding (Fig. 4) scheme was used for optimizing the weights, where at first, GA was used for finding initial weights that were further optimized by using BP and vice versa.

The binary bit-string representation of the weights leads to a precision problem, i.e., how many bits would be sufficient for representing weights and what would be the total length of a chromosome. Moreover, the binary representation is computationally expensive because, in each training iteration, a binary to real-valued mapping and vice versa is required. Hence, it is advantageous to use the real-coded chromosome (Fig. 4) directly [147, 148, 149, 150, 151].

Traditional EA operators are applied on the binary chromosome. Thus, operators, such as bias-mutation, unbiased-mutation, node-mutation, weight-crossover, and gradient-operator, etc., were defined, for operating on the real-valued chromosome [147]. On the other hand, a matrix-based representation of weights, where a column-wise and a row-wise crossover operators were also defined [152]. The GA-based real coded weights optimization outperforms BP and its variants for solving real-world applications [153, 154, 155, 156]. Moreover, an evolutionary inspired algorithm, called differential evolution (DE) [105, 157] that imitates mutation and crossover operator to solve complex continuous optimization problems was found to be performing efficiently for real-valued weight vector optimization [158, 159, 160].

Similar to DE, swarm-based or bio-inspired based metaheuristics directly apply heuristics inspired by nature on a real-valued vector. Hence, they are advantageous in comparison to an EA-based algorithm that needs to simulated mutation and crossover operators for real-valued weight vector [110]. It was found that PSO guides a population of the FNN weight vectors towards an optimum population [161, 162]. Hence, many researchers resorted to working on swarm based metaheuristics for the FNN optimization.

A cooperative PSO, which suggests to splitting a solution vector into $n$ parts, where each part optimized by a swarm of $m$ particles [163, 164]. Thus, an $n\times m$ combinations are constructs an $n$-dimensional composite vector, where each $m$ swarm contributes to the fitness of a solution. Such cooperation between swarms led to a better performance than that of the basic version of PSO. Similarly, a multi-phase PSO was proposed in which particle position was updated only when improvement in location was found; otherwise, the location was copied as-it-is into the next generation [165].

A cultural cooperative particle swarm optimization (CCPSO) approach in which a collection of multiple swarms that interact by exchanging information was proposed by Lin et al. [166]. The CCPSO performed better than BP and GA when it was applied for optimizing a fuzzy neural network. Similarly, a hierarchical particle swarm optimization was used to design a beta basis function neural network [167].

Apart from PSO, there are numerous metaheuristic algorithms among which, some significant metaheuristics were discussed here that were applied for FNN optimization. The continuous version of ACO [168] was efficiently applied to optimize the FNN weight vector [169]. ACO trained FNN was found efficient in solving real-life applications, such as scheduling, prediction, image recognition, etc. [170, 171].

ABC was efficiently applied on weight vector for optimizing the FNNs [172, 173, 174]. Similarly, considering the efficiency of HS algorithm—that has a slow convergence rate, but guarantees a near-optimum solution [117]—many researchers applied HS for optimizing weight vector of the FNNs [175, 176]. Moreover, the efficiency of HS comes from using $m$ many harmonies (weight vectors), and iteratively improvising each harmony by computing new harmony (new solution vectors) using heuristic inspired by music pitch modification [117, 177, 178].

In the past, many other forms of metaheuristics were also used for optimizing the FNNs. For example, the application of FF, CS, GSO, BFO, and CFO algorithms for the FNN weights optimization is available in [179], [180], [181], [182, 183], [184] respectively.

Moreover, a comparative study showed that FF algorithm performed better than that of BP, GA, and ABC for weight vector optimization [185]. In [186] a detailed study explains the application of the local and global metaheuristic algorithm for FNN optimization. For example, local search algorithms like SA, TS, GRAP, VNS [98], estimations of distribution algorithm [187] and global search algorithms like GA, ACO, and memetic algorithm, were examined thoroughly in [186]. Additionally, many researchers studied the performance of metaheuristic algorithms for the training of the FNN and reported that the metaheuristic approaches outperform all the conventional methods by a huge margin [188, 189, 190].

The memetic algorithm supports the hybridization of two or more global metaheuristics for the FNN optimization, which is evident from the following examples. A hybrid GA and PSO approach for optimizing the FNN were proposed in [131], where both GA and PSO were suggested to run over the same population—randomly generated population W of $m$ individuals (the same individual was treated as a chromosome in GA and a particle in PSO). In each generation of GA and PSO, the fitness of each individual was computed. Then, the best performing individuals (top-half) were marked as elites. The elite individuals were copied to next generation and half of the copied elites were optimized using PSO and the remaining half using GA through tournament selection and crossover operation.

Similarly, a PSO and SA based hybrid algorithm for optimizing FNN were proposed in [191], where, in each iteration, each PSO particle was governed by SA metropolis criteria [100] that determined global best particle for PSO algorithm. There are several other hybrid algorithm examples available in the literature: a hybrid PSO and GA [192]; hybrid GA and DE [193]; hybrid PSO and GSO [194]; and hybrid PSO and optimal foraging theory [195].

3.2.2 Architecture plus weight optimization

The basic architecture optimization approach is a cascade correlation learning, which iteratively adds nodes to hidden layer to construct optimum architecture [66]. Moreover, a constructive (add node iteratively) and destructive (prune nodes iteratively) method [196]. However, the constructive and the destructive methods for optimizing architecture are no different from the manual trial-and-error method. Therefore, genetic representation of the FNN architecture as mentioned in Figs. 5, 5, 5 can be used for architecture optimization, which is equivalent to searching optimum architecture from a compact space of FNN topology [197, 198].

Let us discuss the genetic representation of architecture in detail. A direct encoding scheme (Fig. 5) was proposed in [199, 200], where authors used an adjacency matrix (Fig. 4) to represent connections between nodes, where between any two nodes $i$ and $j$, a presence of connection is indicated by “1”, and absence of connection is indicated by “0”. Hence, they were able to encode complete structural information into a chromosome. However, it is disadvantageous because chromosome length increases with network size. Therefore, if only the network’s structural information can be encoded into genotype, then, it will avoid chromosome length problem [201]. Additionally, the encoded network structural information can be accessed using rule-based recursive equation [202]. Moreover, the represented parametric/structural information into the chromosome can indirectly provide access to the rest of the structural details from a predefined archive (parametric information) [203].

The indirect encoding scheme reduces chromosome length, where parametric information, such as the number of hidden layers, the number of nodes at hidden layers, the number of connection, etc., makes an archive $s$. The production rule (Fig. 5) allow us to get access to complete structural information (Fig. 5). Hence, a rule based encoding scheme allows a better FNN architecture optimization than a direct encoding scheme [204].

Unlike the weight optimization that has only limited ways of genetic representation, the FNN architecture optimization is an interesting area of research as there are various ways to represent architecture into genotype. It is evident from a fractal configured FNN representation in [205], where authors define each node using parameters, namely, edge code, input coefficient, and output coefficient. Similarly, in [206], GA was applied to evolve each layer separately and in [207], a grammar encoding and colonial competitive algorithm were proposed.

Another approach to the genetic representation of architecture is to encode weights w (real vector: Fig. 4) and architecture vector $\text{\bf{a}}=\langle a_{1},\ldots a_{m}\rangle$ (binary vector as Fig. 5) into a combined genotype. Hence, a single solution vector $s=\langle w_{1},\ldots,w_{n},a_{1},\dots,a_{m}\rangle$ is obtained [208], which can be optimized by using metaheuristics.

Many researchers improvised the algorithms itself to optimize architecture. Such examples are as follows: in [209], a PSO-PSO method was proposed, in which a PSO (inner PSO block) was applied for optimizing weights that were nested under another PSO (outer PSO block) which was applied for optimizing the architecture of FNN by adding or deleting hidden node. Similarly, in [210, 211], a hybrid Taguchi-genetic algorithm was proposed for optimizing the FNN architecture and weights, where authors used a genetic representation of the weights, but they select structure using constructive method (by adding hidden nodes one-by-one). A multidimensional PSO approach was proposed in [212] for constructing FNN automatically by using an architectural (topological) space. Moreover, the individuals in the swarm population were designed in such a way that it optimized both position (weights) and dimension (architecture) of an individual in each iteration. Thus, optimized FNN weights and architecture simultaneously.

So far, only genetic representation was discussed for evolving architecture. However, GP can optimize a phenotype itself, where genetic representation is not required [213]. Therefore, EP and GP can be directly applied to a population FNN architecture to evolve an optimum FNN architecture [148, 214, 215].

The design of the FNN architecture is responsible for processing high-dimensional data. Hence, deep learning paradigm offers study massive and deep structure of the neural network that can process complex problems related to speech processing, natural language processing, signal processing, etc., [216, 217]. Such a variant of the FNN is convolutional neural networks (ConvNets), which is designed to process data from the multiple arrays form such as a color image composed of three 2D arrays [216, 217]. The ConvNets has a three-dimensional arrangement of neural nodes. Hence, it efficiently receives 3D inputs and processes them to produce 3D outputs [218].

In contrast to deep network paradigm, an extreme learning machine (ELM) based hierarchical learning framework (H-ELM) proposed in [219] claimed a faster learning than deep learning by ELM [220] based auto-encoding. The proposed H-ELM framework worked in two phases: unsupervised hierarchical feature representation and 2) supervised feature classification [219].

3.2.3 Input layer optimization

Input layer optimization resembles feature reduction, which is traditionally performed separately by dimensionality reduction methods [221]. However, reducing input dimension by optimizing input layer, i.e., by feeding a subset of input features at the input layer than by feeding the whole set of input features enhances FNN’s performance [222, 223]. Therefore, FNN has a functional dependency on the problem at hand.

EAs select a subset of input features for which FNN perform better than that of the complete feature set [222]. For this purpose, a genetic representation of input features is required in which the available features are placed on a genetic strip, and the presence of a feature is marked as “1” and the absence of a feature is marked as “0.” Such mechanism of input layer optimization was found advantageous in improving NN performance [224].

Moreover, binary PSO [225], which is a discrete optimization method was employed for selecting the input features which were binary coded [226, 227]. In [226], a modified version of binary PSO was proposed where EA like mutation operator was applied to mutated binary vectors. Similarly, ACO, which traditional solves discrete optimization problem was applied to select input features and training of an FNN in a hybrid manner [228].

Input layer optimization which is related to input feature reduction can also be thought as training data optimization. Training data optimization is helpful, particularly when data is insufficient or noisy. In [229], an adaptive selection of input examples was performed by employing genetic selection, where two-point and one-point crossover operations created new example patterns. For the crossover operations, the parent’s examples were drawn from the original input set. Also, mutation operators were also applied for generating new child example. Hence, the efficiency of FNN was improved when trained over the modified new examples.

Additionally, in [230], an input example generation methods was proposed in which the input space was divided into many regions, and $k$-nearest neighbor method was applied to determine/generate a new virtual example, mainly for the sparse region of the input space. Hence, both the above methods of input example generation sought to enrich knowledge space for the FNN learning [229, 230].

3.2.4 Node optimization

Primarily, node optimization can be addressed in three ways: 1) by choosing activation functions at the FNN active nodes from a set of activation functions [231, 156]; 2) by optimizing the arguments of activation function [232]; and 3) by placing a complete model at the nodes of a network [233, 234].

It was found that FNN performed better when it has non-homogeneous nodes (different activation function at different nodes) than that of the homogeneous nodes [235]. In [231], evolution in FNN nodes were offered by selecting sigmoid and Gaussian function adaptively at the nodes [231]. Moreover, adaptation in both nodes and architecture using EAs, where the design of nodes was inspired by locus flight system and tailflip of crayfish [236], can further improve FNN performance [237]. For this purpose, in [238], an input dependent FNN that had a combined chromosome representation (Fig. 6) was proposed, where a real-coded GA for simultaneous optimization of weights, activation functions, and architecture was used.

On the other hand, to optimize nodes, a family competitive EA was proposed in [239], where three operators, such as decrease-Gaussian-mutation, Cauchy-mutation, and self-adaptive-mutation were defined. Moreover, family-competition is a process that generates a pool of $L$ many FNNs by recombination and mutation operations and selects an FNN from that pool. The family-competition with different mutation operator is repeated until the best FNN is found. Many others found that the adaptation in FNN nodes by one of the methods mentioned above can improve FNN performance to some extent [240, 241, 242, 243].

The third aspect of node optimization is to design a node as a model itself. Such modification leads to variate of neural network paradigms such as polynomial neural network [233, 244], where the nodes are designed to as a polynomial function based on inputs to the nodes. Similarly, the nodes of a GMDH neural network is designed as an Ivakhnenko polynomial [245]; the nodes of a complex value neural network or multivalued neural network is designed with a complex value activation functions [234]; the node of spiking neural networks has specific behavior, in which a node signal is propagated to another node only if the intrinsic quality of neural activation value is above a defined threshold [246]; the nodes of fuzzy neural network paradigm is designed using the concepts of fuzzy theory [247]; the node and the architecture of the Quantum neural network are inspired by the quantum computing [248, 249, 250, 251]. In all such methods, metaheuristics have a significant role in the optimization.

3.2.5 Learning algorithm optimization

The initial thought of learning algorithm optimization is the optimization of its parameters. For example, the optimization the learning rate and the mutation factor parameters of BP by applying some metaheuristics [146]. To optimize the parameters of an FNN learning algorithm, its parameters (e.g., BP parameters) and learning rules are encoded onto a genotype [201, 252]. However, formulating BP parameter such as learning rule, which is a dynamic concept, into a static chromosome is disadvantageous [253]. Hence, a genetic coding for four components (current weight, activation function of the incoming node and outgoing nodes, input) local to weight in an FNN can encode [253]. Moreover, assuming that each node in a network uses same learning rule, an evolution in learning was proposed in [254], where weights optimization related to a particular node depended only on the input/output at that node. Evolution in learning rule can be described as [255]:

[TABLE]

where $t$ is the iteration, $\Delta\text{\bf{w}}^{t}$ is weight change, $w_{1},w_{2},\ldots,w_{n}$ are weights associated with a node, $\theta_{i}$ is real-valued coefficient that is determined by using EAs. However, learning rule (10) is impractical because of it required huge computation time. Hence, bio-inspired algorithms may be employed for determining the coefficient in (10).

3.2.6 Combination of FNN components optimization

Fig. 3 is an impressive representation of the most of FNN optimization combinations, where the genetic representation of the combination of FNN components can be represented in Fig. 6, which refers to a hierarchy of combination, called meta-learning scheme. In the meta-learning scheme, top down or bottom up optimization approach, which means, weights to learning rule and learning rules to weight optimization can be adopted [256]. However, it resembles one-by-one learning scheme. Hence, the advantageous approach is to represent each component of FNN side-by-side onto a genetic vector for optimization, which indicates the confluence of all components of FNN as indicated by area “a8” in Fig. 3.

Yao [132, 255] summarized all such form of adaptation in evolutionary artificial neural network (EANN), which is a special class of artificial neural network, where in addition to learning; evolution is another fundamental form of adaptation. Infact a paradigm, called Neuroevolution that accommodates adaptive learning all or some components of FNN in some intuitive ways by applying EAs. For examples, generalized acquisition of recurrent links (GNARL) [257], evolutionary programming net (EPNet) [258], neuroevolution of augmenting topologies (NEAT) [259], hypercube-based neuroevolution of augmenting topologies (HyperNEAT) [260], evolutionary acquisition of neural topologies (EANT2) [261], and heterogeneous flexible neural tree (HFNT) [262] optimizes both FNN structure and parameters (weights) using some direct or indirect encoding methods. Moreover, several other paradigms and methods proposed in the past for the simultaneous optimization of FNN components are described as follows.

A structured genetic algorithm was proposed in [263], which simultaneously optimized both architectures and weights. It was found that the simultaneous optimization of both weight and architecture lead to a better generalization  [264, 197, 56, 265]. Considering permutation222Permutation problem occurs when using traditional crossover operator, where a population has traditional genetic representation of FNN architecture. problem in a GA, EP-based mutation mechanism for evolving FNN architecture was proposed in [258] is known as EPNet.

A neuroevolution of augmenting topologies (NEAT) introduced in [259] was a GA-based evolution of an FNN phenotype as a whole, in which a special mutation and crossover operator were defined for manipulating nodes and connections of FNN. Specifically, the linear network information FNN weights, nodes, and connection information were encoded using genetic encoding. The proposed NEAT was evaluated over several applications, and its performance was found outperforming static FNN topology.

A virtual subpopulation approach was proposed in [266] for the optimization of FNN using EAs. Later, while indicating a permutation problem, crossover operator as a combinatorial optimization problem was proposed in which each hidden node was considered as a subnetwork and a complete network was evolved using the evolution of several subnetworks [267]. Additionally, GA-based and SA-based crossover operators were applied to generate an offspring (new individual subnetwork). To maintain diversity in population, two mutation operators such as BP-mutation and random-mutation were proposed. In BP-mutation, few iterations of BP algorithm were applied to update weights of the subnetwork, and in random mutation, weights of subnetwork were randomly replaced with new weights. Hence, a coevolution of FNN weights and architecture was proposed that evolved FNN with the cooperation of the individuals of a subnetwork population.

A cooperative coevolution neural network process—inspired by virtual subpopulation approach [266]— was proposed in [268], which was a symbiotic, adaptive neuroevolution (SANE) algorithm for constructing FNN in a dynamic environment. Unlike conventional evolutionary approach, which uses a population of FNNs, SANE uses a population of nodes, where each node establishes connections with the other nodes to form a complete network.

Two reasons of better performance of SANE over conventional and stand-alone metaheuristics were suggested. First, since SANE consider the nodes as functional components of the FNN, it accurately searches and evaluates nodes as genetic building blocks. Second, since a node alone cannot perform well and evolutionary process evolves different types of nodes, SANE was able maintains diversity in the population. Later, the concept of SANE was extended, in which the selection of several individuals from a population of hidden nodes was combined in a various permutation in order to form several complete networks, i.e., evolution in hidden nodes led to an evolution of the complete network [269].

A concept of sparse neural trees, in which GP for evolving network structure and GA for parameter optimization was proposed in [270]. Similarly, a flexible neural tree (FNT) concept, where GP was used for the adaptation in network structure and SA for the optimization of the parameters (including parameters of activation function) was proposed in [271, 272]. FNT is a tree-like model where adaptation in all components of is equally important. Moreover, its components adaptation may take many forms (Fig. 3). Hence, a beta basis function—which has several controlling parameters, such as shape, size, and center—was used at non-leaf nodes of an FNT [273]. It was observed that embedding beta-basis function at FNT nodes has advantages over other two parametric activation function. A parallel evolution of FNT using MPI programming and GPU programming respectively were proposed in [274] and in [275].

A slightly different direction of FNN modification and improvement study can be seen as the study of quantum neural network (QNN). At the first place, the QNN as a quantum perceptron was proposed by Lewestein [276], where instead of classical weights, a unitary operator was used to map inputs to an output. The study in QNN encompasses the development of quantum weights, quantum neurons, quantum network, and quantum learning [249].

The design/algorithm of quantum network was thought as an algorithm that can find the control parameters for a coupled qubit system [277] as it appears in quantum computing. A comprehensive quantum inspired neural network is presented in [278], where two categories of inspiration were drawn: strongly and weakly quantum inspired FNN. In strongly inspired QNN, each pattern in a training set was considered as a particle which is processed by a number of FNNs in different universes. Such process was compared with the electrons or photons passing through many slits simultaneously. Whereas, in weakly inspired QNN, each training pattern (though as a particle) was in its own universe. Moreover, there were various QNN models proposed in the past by 1) Behrman et al. [279], 2) Chrisley [280], 3) Menneer [278], and Ventura [281]. A detailed description of these QNN models can be found in [282].

3.3 Comments on metaheuristics approaches

It is indeed can be concluded that metaheuristic algorithms have provided various dimension to the optimization of the FNNs. It has opened several ways such that a generalized FNN can be obtained. Especially the architecture simplification which is directly related to the generalization of an FNN can easily be achieved through the evolving FNN together with its other components. However, the primary disadvantage of using metaheuristic algorithms is the training time consumption. Since the metaheuristic algorithms use a population (many solution candidates) during optimization, the time consumption becomes directly proportional to the number of candidates in a population.

It can also be argued that both conventional and metaheuristic based FNN training take by far more training time than the extreme learning machine (ELM) [283]. ELM is a three-layered FNN architecture whose weights between input and hidden layer are randomly assigned and never updated. Additionally, the weights between hidden and output layer are updated in a single step using some least square estimation. Hence, extremely less time required for the learning of the FNN.

It is stated in NFL theorem [92] that it is difficult to find a metaheuristic algorithm that solves all class of problems. Hence, a metaheuristic algorithm may find difficulty in optimizing the FNN that has been formulated for solving some specific problem (input patterns). Additionally, it is not theoretically possible to understand or determine that how fast a metaheuristics algorithm will converge or finds a satisfactory solution. The only way to determine a metaheuristic algorithm’s convergence is by its empirical evaluations. Moreover, since each metaheuristic applies some specific heuristic, it is difficult to select one metaheuristic as the best metaheuristic at an instance for a problem. It is only possible to select a metaheuristic by empirically comparing the convergence speeds and trained FNN performances.

4 Multiobjective metaheuristic approaches

Multiobjective optimization procedure involves in optimizing two or more objectives functions, simultaneously. Multiobjective algorithms are efficient methods for evaluating Pareto-optimal solutions for multiobjective problems. Since optimizing training error cannot provide generalization alone, FNN optimization is viewed from the multiobjective perspective.

Let us first investigate: why the multiobjective framework is needed for FNN optimization, what are the objective functions required for framing FNN as a multiobjective problem, and how the objective functions can be framed into multiobjective optimization. Answers to these question lie in the following discussion.

First, a cost function (4) or any equivalent function is the foremost necessity for the supervising training of FNN.

Second, the generalization of FNN is an essential aspect of its optimization. One approach is to use validation error on cross-validation data because an FNN with low training error may not perform well on unseen (test) data unless FNN is generalized. Moreover, minimization of generalization error is essential than the minimization of training error.

Another approach is to add a regularization term to the training error to avoid overfitting. Additionally, minimizing network complexity leads to a better generalization [284]. Hence, generalization can be achieved by adding a complexity indicator term to training error, i.e., the generalization by minimizing training error and simplifying network complexity.

Third, reducing input-feature—when a problem is available with a huge input dimension (feature)—can lead to a better generalization. However, input dimension reduction and training error reduction are two contradictory objectives.

Finally, the conclusion is, the training error (4) or equivalent cost function $c_{f}$ needs to be optimized with one or more additional objectives to achieve generalization, which is why multiobjective framework for optimizing FNN are used.

Let us say that training error (4) is $c_{trn}$, and an additional objective is $c_{add}$. So, a generalized error $c_{gen}$ may be computed by adding an objective to training error as:

[TABLE]

where $\lambda>0$ is a hyperparameter that controls the strength of additional objective $c_{add}$. The validation error term $c_{cv}$, regularization term $c_{reg}$, or network complexity $c_{net}$ or a combination of all can be considered as an additional objective $c_{add}$ in (11). The regularization term $c_{reg}$ is the weight decay or norm of weight vector w as:

[TABLE]

Similarly, a validation error $c_{cv}$ is usually computed using (4) on a cross-validation data. On the other hand, the network complexity $c_{net}$ is computed as:

[TABLE]

where $z$ is the number of nodes in the network $c$ (Fig. 5), or any user-defined function can also be used for evaluating network complexity, e.g., the number of nodes, the number of connections, etc.

However, the generalization objective of the form (11) is a scalarized objective that has two disadvantages [285]. First, determining an appropriate hyperparameter $\lambda$ that controls the contradicting objectives. Hence, the generalization ability of the produced model by using (11) will be a mystery. Second, the objective (11) leads to a single best model that tells nothing about how contradicting objectives were handled. In other words, no single solution exists that may satisfy both objectives. Therefore, generalization error (11) need to be formulated into a multiobjective form: $\min\{c_{trn},c_{reg},c_{cv},\ldots\}$, i.e., a multiobjective optimization needs to be performed as:

$\mbox{minimize }\{c_{1}(\text{\bf{w}}),c_{2}(\text{\bf{w}}),\ldots,c_{m}(\text{\bf{w}})\}$

$\mbox{subject to }\text{\bf{w}}\in S,$

where $m\geq 2$ is the number of objective functions $c_{i}:\mathbb{R}^{n}\to\mathbb{R}_{\geq 0}$. The vector of objective functions is denoted by $\text{\bf{c}}=\langle c_{1}(\text{\bf{w}}),c_{2}(\text{\bf{w}}),\ldots,c_{m}(\text{\bf{w}})\rangle$. The decision (variable) vectors $\text{\bf{w}}=\langle w_{1},w_{2},\ldots,w_{n}\rangle$ belong to the set $S\subset\mathbb{R}^{n}$, which is a subset of the decision variable space $\mathbb{R}^{n}$. The word “minimize” indicates the minimization all the objective functions simultaneously.

A nondominated solution is one in which no one objective function can be improved without a simultaneous detriment to at least one of the other objectives of the solution. The nondominated solution is also known as a Pareto-optimal solution.

Definition 3**.**

Pareto-dominance - A solution $\text{\bf{w}}_{1}$ is said to dominate a solution $\text{\bf{w}}_{2}$ if $\forall i=1,2,\ldots,m$, $c_{i}(\text{\bf{w}}_{1})\leq c_{i}(\text{\bf{w}}_{2})$, and there exists $j\in\left\{1,2,\ldots,m\right\}$ such that $c_{j}(\text{\bf{w}}_{1})<c_{j}(\text{\bf{w}}_{2})$.

Definition 4**.**

Pareto-optimal - A solution $\text{\bf{w}}_{1}$ is called Pareto-optimal if there does not exist any other solution that dominates it. A set Pareto-optimal solution is referred to as a Pareto-front.

Now, a multiobjective algorithm must provide a homogeneous distribution of a population along Pareto front and improve solutions along successive generations [286]. Hence, three basic operators can be used [286].

1. Fitness assignment to guide a population in the direction of Pareto-front using robust and efficient multiobjective selection method.
2. Density estimation to maintain solutions distributed over entire Pareto-front using operators that take account of the solution’s proximity.
3. Archiving to prevent degradation in fitness during successive generations by maintaining an external population to preserve the best solutions and for periodic input to the main population. A detailed survey of multiobjective algorithms is available in  [287, 288]. Now, based on the above discussion on the cost functions and generalization conditions, the multiobjective for FNN optimization can be categorized as non-Pareto based multiobjective approach and Pareto-based multiobjective approach.

4.1 Non-Pareto based multiobjective approaches

In non-Pareto based multiobjective approach, the objective functions are aggregated as mentioned in (11) or by some other means. For example, in [289], authors proposed to add a regularization term $c_{0}-c_{reg}$ to training error $c_{0}-c_{trn}$, where $c_{0}$ is the origin of the two objectives. To obtain an efficient solution, they designed a vector vc of scalar objectives by varying the hyperparameter $\lambda$ from 0 to 1. Hence, training FNN for each scalar objective in vector vc, a Pareto set was obtained, and then, it was possible to select the best solution from the Pareto front. However, this was an expensive approach, which does not use any Pareto-based multiobjective algorithm to compute Pareto set; rather, an ellipsoid method [290] was applied to train FNN for each scalar objective $vc_{i}\in\textbf{vc}$ sequentially.

Similarly, in [291], to achieve generalization, authors proposed a sliding mode control BP algorithm for the multiobjective treatment to FNN objectives $c_{trn}$ and $c_{reg}$. The optimization trajectory of the 2D space of the objectives $c_{trn}$ and $c_{reg}$ was controlled by modifying BP weight update rules using two sliding surface control indicators each belongs to the mentioned objectives, respectively

Multiobjective treatment to FNN was also offered by using improvising metaheuristics itself such as a predator-prey algorithm was proposed in  [292]. To get a generalized network, the predator-prey algorithm used a family of the randomly generated population of sparse neural networks, called pray population and an externally induced family of predators population whose job was to prune preys populations based on the objectives $c_{trn}$ and $c_{net}$ was also generated. Similarly, a hybrid multiobjective approach, where a geometrical measure based on singular-value-decomposition for estimating a necessary number of nodes in a network was proposed in [293].

Additionally, a micro-hybrid genetic algorithm was introduced to fine-tuning the network performance. A hybrid algorithm, which uses GA for evolving FNN and uses PSO, BP, and LM for fine-tuning the evolved FNN was proposed in [294]. In the proposed hybrid algorithm, several objectives function such as training error $c_{trn}$, validation error $c_{cv}$, number of hidden layers $c_{hid}$, number of nodes $c_{node}$, and activation function $c_{fun}$ were aggregated as:

[TABLE]

where, $\alpha$, $\beta$, $\gamma$, $\delta$, and $\theta$ were controlling parameters. Hence, multiple objectives were optimized simultaneously.

As mentioned above in Section 4, the aggregating objective function has disadvantages in obtaining the best generalized solution. It is evident from (14) that determining hyperparameters for controlling objective function is a challenging task. Therefore, Pareto-based multiobjective is an efficient choice for the multiobjective treatment of FNNs.

4.2 Pareto based multiobjective approaches

The advantages of applying Pareto-based learning is thoroughly explained and compared with a single and scalerized objective in [295]. For example, a nondominated sorting genetic algorithm version II (NSGA-II) [296] when used for optimizing objectives $c_{trn}$ and $c_{net}$ offers a Pareto set by optimizing both objectives simultaneously using a nondominated sorting method as defined in Definition 3. Hence, NSGA-II can be applied to obtained a regularized network by optimizing the objectives $c_{trn}$ and $c_{reg}$ [297].

Similarly, Pareto differential evolution (PDE) algorithm and its variant self-adaptive PDE algorithm was applied to optimize objectives $c_{trn}$ and $c_{net}$ simultaneously that offered a Pareto-set, from which the best solution was picked-up according to network complexity and approximation error examination [298, 299]. Simultaneous optimization of the objectives $c_{trn}$ and $c_{net}$ were also addressed using multiobjective PSO to generalize FNN performance [300].

For an image classification problem, authors in [301], pointed out two crucial points: the classification speed and the classification accuracy $c_{acc}$. The classification speed was then related to the network complexity (number of hidden neurons) $c_{net}$. The proposed trade-offs between classification speed and classification accuracy were addressed using NSGA-II.

Similarly, in [302], authors studied three methods for image classification problem: linear aggregating (LA), NSGA-II with deterministic selection (DM), and NSGA-II with tournament selection (LM). They proposed to optimize network complexity $c_{net}$ and accuracy $c_{acc}$. Moreover, they combined regularization term $c_{reg}$ with accuracy $c_{acc}$ and proposed an adaptive strategic for designing network topology using reproduction operators for both hidden layer and input layer. The hidden layer operators were add-connection, delete-connection, add-node, and delete-node. The receptive (input) layer had the following operators: add-connection, delete-connection, add-node, and delete-node. Interestingly, they observed that DM and LM performed better than LA, i.e., Pareto-based multiobjective algorithms performed better than that of the scalerized objectives. Such ability of the Pareto-based treatment to FNN to obtain general FNN was exploited by several researchers for solving many real-life applications [303, 304, 305, 306].

Further, the coevolution FNN concept [269, 307] was extended in [308] under the multiobjective framework, by using subnetwork and network concepts. A subnetwork was a collection of nodes, i.e., a subnetwork was considered as a hidden node for a network. Therefore, a network was a collection of subnetworks. So, a population $P_{1}$ of subnetwork, which was evolved separately using NSGA-II was used to construct a population $P_{2}$ of networks. Then, NSGA-II was again applied to evolve population $P_{2}$. Interestingly, authors defined separate objectives for population $P_{1}$ (subnetworks objectives) and $P_{2}$ (networks objectives) so that the functional diversity in both network and subnetwork can be maintained. Additionally, some metrics (objectives) for measuring network and subnetwork functional diversities were defined. The objective of subnetworks were differences (for maintaining functional diversity of subnetwork), substitution (to replace poor candidates by better candidates), and complexity (for counting the number of connection, nodes, and sum of all weights). Therefore, they coevolved overall network with the cooperation of subnetwork that evolves together with the whole network to get a general solution to a problem.

Apart from the discussed objective in this section, some interesting dimensions in multiobjective treatment to FNN can be noted in [309], in which authors proposed to apply NSGA-II for the simultaneous optimization of three objectives: input-dimension, training error, and network complexity. Hence, an optimized a network that performs well on the minimal set of input dimension was obtained. Similarly, in [310, 41], authors used a Pareto-based memetic algorithm approach for combining PDE and Rprop algorithms to minimize objective pairs true classification rate and minimum sensitivity (miss-classification rate), simultaneously.

As a result of metaheuristic or multiobjective metaheuristic treatment, a set of FNN network is obtained and selecting the best FNN from that set is a difficult task. Since selecting a single best FNN from may not offer a generalized solution and the residual error can still be remaining many problems when selection single best FNN [311], then an ensemble of a set of FNNs is recommended.

5 Ensemble of feedforward neural networks

Metaheuristics optimization of FNN leads to a final population that contains many solutions close to the best solution. Moreover, the solution in the final population are divers in the following sense: 1) parametric (each FNNs have different sets of weights); structural (each FNNs have different network configurations); and 3) training set (each FNNs are trained on different parts of a training set). Hence, a collective decision (ensemble) of $l$ many diverse candidates selected from a final population may offer desired generalization [312]. The literature that explains how to construct diverse FNNs and how to combine decisions of diverse FNNs are summarized as follows.

The very basic idea is to apply single-solution based algorithms on $l$ many FNNs to get $l$ many diverse solutions [46]. The decision of $l$ many candidates which were created either by single solution-based metaheuristics, or by population-based metaheuristics, or by any other means are combined using the following methods [313, 314]: 1) majority voting method (for classification problems); 2) arithmetic mean (for regression problem); 3) rank-based linear combination; 4) linear combination by using recursive least square [315] (to minimize weighted least squares error so that redundant individuals are eliminated); 5) evolutionary weighted mean or majority voting (metaheuristic to determine impact of an FNN in ensemble); and 6) entropy-based method for combining FNNs in ensemble (assigning entropy to FNNs during the learning process) [316].

Since population-based metaheuristics lead to an optimized final population, it is advantageous to use the final population for making ensemble [312]. However, there are two fundamental problems with it [317]: 1) determining ensemble size, 2) how to maintain diversity in the population. Hence, a negative correlation learning (NCL) algorithm that optimized and combined individual FNNs in an ensemble during learning process was proposed in  [317]. NCL optimized all individual FNNs simultaneously and interactively by adding a correlation penalty terms to the cost functions. Moreover, NCL produced negatively correlated and specialized FNNs by using co-operation among each FNNs of a population [318].

To determine the size of ensemble automatically, EA-based ensemble procedure was laid down in which NCL was applied during networks training. Moreover, different FNNs were allowed learn different parts of training data and the best (according to fitness) were selected for ensemble [319]. Additionally, a constructive-cooperative-neural-network-ensemble was proposed in [320] that determined ensemble size by focusing on accuracy and diversity during a constructive, cooperative procedure [321].

However, mere training fitness based selection of candidates for the ensemble is insufficient because it does not tell much about candidates role/influence in the ensemble. This problem was addressed in a GA-based selective ensemble method [322], which selects a subset of the population and determine the strength of selected candidates using GA. It was also shown that the ensemble of a subset of the population was found performing better than that of the whole population [322]. The effectiveness such GA-based selection was found efficient than the traditional ensemble methods: bagging [323] and boosting [324].

It is beneficial to partition/fracture training data and allows different FNN in the population to learn various parts of training data [323, 324]. An evidence of such was examined in [325, 326], where it was found that the ensemble of a few FNNs that was trained using bootstrapping performs better than that of an ensemble of a larger number of FNNs. Similarly, the efficiency of using distinct training sets for optimizing different FNNs was proved when a class-switching ensembles approach proposed in [327] and were compared with bagging and boosting methods.

At one hand bootstrapping method allows FNN to learn different training samples. On the contrary, a clustering-and-coevolution approach for constructing neural network ensembles proposed in [328] partition the input space using a clustering method to reduced number of input nodes of FNNs. Hence, in the ensemble, diverse FNNs (different FNNs were specialized in various regions of input space) were created. Moreover, it reduced run time of learning the process by coevolving (divide-and-conquer method) different FNNs using cooperation between FNNs. Such method improves diversity and accuracy of an ensemble system [328].

Similarly, a method was suggested in [329] for generating diverse evolutionary FNNs using a fitness-sharing method—a fitness sharing method shares resources if the distance between the individuals is smaller than the predefined sharing radius. Specifically, authors proposed a speciation based evolutionary neural ensemble method for constructing ensemble by combining FNNs using a knowledge space method. On the other hand, a progressive interactive training scheme called a sequential-neural-network-ensemble-learning method, which trained FNNs one-by-one by interaction from a central buffer of FNNs was proposed in [330].

Both diversity and accuracy is a crucial aspect in construing ensemble of FNNs [313]. However, accuracy and diversity are contradictory to each other, so, a multiobjective approach may be applied to evolve FNN population by maintaining accuracy and diversity simultaneously [331]. For this purpose, multiobjective regularized negative correlation learning that maximized performance and maximized the negative correlation between individuals in population was found efficient [332].

6 Challenges and future scopes

The effectiveness of FNN training primarily depends on data quality, which is governed by the following data quality assurance parameters: accuracy, reliability, timeliness, relevance, completeness, currency, consistency, flexibility, and precision [333, 334]. Usually, data cleaning is a major step before modeling [335]. Therefore, training of the FNN remains always sensitive to the data cleaning process and it poses a significant challenge to adapt some mechanism in training process such the sensitivity towards data-clean may be reduced. Additionally, one problem related to data-driven modeling (FNN learning) is the data itself which can be insufficient, imbalanced, incomplete, high-dimensional, or abundant.

For the case insufficient data, usually the input hyperspace is exploited to generate virtual samples to fill the sparse area of the hyperspace, and by monitoring FNN performance on the virtually generated samples [336]. The second approach exploits the dynamics of EAs in conjunction with FNNs to obtain new samples [229]. However, this area is still much to explore, where some open questions such as how efficiently FNNs can be trained with virtually generate data to mitigate the insufficiency. On the other hand, research in the area of imbalance dataset is continued to interest researcher [337].

The present era of data analysis is what we call big data, i.e., we need to deal not only with high-dimensional data but also with the variety data and stream data [338]. High-dimensional data, such as gene expression data, speech processing, natural language processing, social-network-data, etc., poses significant challenges. Such challenge is to some extent addressed by deep learning paradigms that allow the arrangement several units/layers of FNNs (or any other model) in a hierarchical manner to process and understand insights of such high-dimensional data [339, 340]. High-dimensional data can also be managed/reduced by encoding or decoding methods and using FNNs training [341]. Therefore, FNNs has a greater role in feature reduction.

In a non-stationary environment, such as stock-price market, weather forecasting, etc., data comes in the stream, i.e., data comes in sequential order, and traditionally, re-training based mechanics for dynamic learning (online learning) of FNN is the basic option [342]. However, it is still an open problem to design strategies for the dynamic training of FNN.

Apart from the crucial aspects that how to manage non-stationary data, the aspect that how to handle multi-view (heterogeneity) of data is an additional challenge. The quest of developing a model that can stand robust and efficient for the non-stationary data caused by the time-dependent process of data generation, and can accommodate new knowledge (newly generated data sample) is a significant topic in machine learning research [343]. On the other hand, integration of data or of the models for that matter for the heterogeneous data generated or gathered from different instruments and data-generation processes is a significant research problem [344, 345].

Moreover, present era, the fourth industrial revolution, is of Internet of Things (IoT) [346]. In IoT, sophisticated technologies such as smartphone and smartwear provide several forms of data, e.g., human activity recognition [347]. Additionally, it demands application to be simple. Hence, FNN models which when aims to such technologies needs to be less complex. Therefore, FNN architecture simplification or model’s complexity reduction is a challenging task. Such problem can be addressed through the integration of FNN with statistical methods like the one usually done with hidden Markov model[348]. Therefore, such kind of modification to network architecture and specialized node design may lead to different paradigms of FNN that may solve various real-world complex problems.

7 Conclusions

Feedforward neural network (FNN) is used for solving a wide range of real-world problems, which is why researcher investigated many techniques/methods for optimizing and generalizing FNN. Specifically, metaheuristics allow us to innovate and improvise methods for optimizing FNN that in turn address its local minima and generalization problems.

Initially, only gradient based linear approximation and quadratic approximation methods for optimizing FNNs were employed to train FNN. These conventional algorithms (backpropagation, Quickpro, Rprop, conjugate gradient, etc.) are local search algorithms that exploit current solution to generate a new solution; however, they lack in exploration ability, therefore, usually, finds local minima of an optimization problem.

Unlike conventional approaches, metaheuristics (e.g., genetic algorithm, particle swarm optimization, ant colony optimization, etc.) are good at both exploitation and exploration and can address simultaneous adaptation in each component of FNN. However, no single method can solve all kinds of problem. So, we need to improvise, adapt, and construct hybrid methods for optimizing FNN. Therefore, several dynamic designs of FNN are reported in the literature: EPNet (an adaptive method of FNN architecture optimization), neuro-evolution of augmenting topologies, flexible neural tree, cooperative coevolution neural network, etc., are among them. Hence, there is a wide spectrum of FNN optimization/adaptation is possible with metaheuristic treatment to FNNs (Fig. 3) in which the fundamental aspect is the formulation of FNN (phenotype) to vectored form (genotype) or any other form of mechanism for manipulation of FNN components.

Since there are many components to be manipulated by means of metaheuristic strategies and the availability of the fact that FNN generalization ability depends on the optimization its all the components, multiobjective treatment to FNN were used. The multiobjective-based training allows an FNN to evolve with handling two or more FNN-related objectives, such as approximation error, network complexity, input dimension, etc. Moreover, the generalization ability of system can be easily improved by combining decision of many candidates of the system. Hence, an ensemble of FNNs by making use of the metaheuristic final population was proposed and the two crucial aspect accuracy and diversity of an ensemble were taken care during propose of evolving FNNs.

It is evident from such aspects of FNN optimization that the future research will be able to bring the new paradigms of FNNs by applying or by the inspiration from the discussed methods in this article. Hence, that will overcome the data quality problem and will be handling new challenges of big data to cope-up with the new era information processing.

Acknowledgment

Authors would like to thank the all the anonymous reviewers for the technical comments, which enhanced the contents of the preliminary version of this paper. This work was supported by the IPROCOM Marie Curie Initial Training Network, funded through the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme FP7/2007–2013/, under REA grant agreement number 316555.