Globally Coupled Particle Swarm Optimization
Liguo Yuan

TL;DR
This paper introduces GCPSO, an enhanced particle swarm optimization method that integrates globally coupled map lattices to improve solution search capabilities by making each particle influenced by all others.
Contribution
The paper proposes a novel GCPSO algorithm combining GCML with PSO, where each particle's speed is influenced by all particles, enhancing search performance.
Findings
GCPSO outperforms traditional PSO in solution search ability.
The influence of all particles improves convergence speed.
Experimental results demonstrate stronger optimization capabilities.
Abstract
All things in the world are interconnected, the only difference is the strength of their connections.Particle swarm optimization(PSO) simulates the foraging behavior of a flock of birds, information is transmitted to quickly find the location of food. This is a process of information exchange, where birds influence each other, constantly adjusting their position and speed values,and updating the optimal position information.In this paper, we propose a globally coupled particle swarm ptimization(GCPSO) that combines the globally coupled map lattices(GCML) with the PSO to enhance its optimization capabilities.The information of the -th lattice point is influenced by the information of all lattice points in GCML.The information between lattice points is interdependent.Inspired by this, we will integrate GCML into PSO and propose a new improved particle swarm optimization, namely…
| Algorithm Globally Coupled Particle Swarm Optimization |
| Initialization of the population position , velocities and parameters , , , , , , , etc; |
| Compute the objective function value (1) of each initialization position ; |
| Identify the global best value (), and previous best value () of each |
| particle, where , and ; |
| while termination criteria (Maximum number of iterations) do |
| for each particle and velocity do |
| (i) Based on equation (12), update speed velocity ; |
| (ii) Based on equation (13), update speed velocity ; |
| end for |
| With updated solutions , calculate the objective function value; |
| Update the positions of global and individual optimal solutions once again, |
| i.e., and |
| end while |
| Return the optimal value as the optimal solution of the optimization problem (1). |
| Until stopping condition is true. |
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMetaheuristic Optimization Algorithms Research · Advanced Multi-Objective Optimization Algorithms · UAV Applications and Optimization
Globally Coupled Particle Swarm Optimization
\nameLiguo Yuana CONTACT L. G. Yuan. Email: [email protected] aDepartment of Mathematics, College of Mathematics and Informatics, South China Agricultural University, Guangzhou, China.
Abstract
All things in the world are interconnected, the only difference is the strength of their connections. Particle swarm optimization(PSO) simulates the foraging behavior of a flock of birds, information is transmitted to quickly find the location of food. This is a process of information exchange, where birds influence each other, constantly adjusting their position and speed values, and updating the optimal position information. In this paper, we propose a globally coupled particle swarm optimization(GCPSO) that combines the globally coupled map lattices(GCML) with the PSO to enhance its optimization capabilities. The information of the -th lattice point is influenced by the information of all lattice points in GCML.The information between lattice points is interdependent. Inspired by this, we will integrate GCML into PSO and propose a new improved particle swarm optimization, namely GCPSO. Here, the speed update formula has been modified and improved. The next flight speed of each bird is influenced by its current speed, its historical best position, the historical global best position, its current position, and the positions of all birds. Each bird(particle) is influenced by all birds. The strength of the impact will be distinguished by the size of the weight. This is the essential difference and key improvement from PSO. Through extensive experiments, it has been found that compared to PSO, GCPSO has a stronger ability to search for solutions.
keywords:
Particle swarm optimization; Coupled map lattices; Globally coupled particle swarm optimization; Benchmark function
††articletype: ARTICLE TEMPLATE
1 Introduction
Optimization problem is the process of finding the optimal solution, with regard to a given criterion, from a set of available solutions. The global optimization problem is usually defined as
[TABLE]
where and The objective function is assumed to be continuous and differentiable [3, 4]. Optimization algorithms require gradient information which is depicted as , then the differentiability of is a necessary condition. However, swarm intelligence optimization algorithms(such as PSO [1, 2]) break these limitations. can be non-differentiable or even discontinuous. Optimization problems arise in various quantitative disciplines. There is a wide range of problems in the relevant literature that can be treated as global optimization problems and these problems can be successfully solved by PSO. Anula Khare et al. reviewed the PSO and successfully applied it in the optimization problem of Solar Photovoltaic system [5]. Houssein et al. conducted a rigorous and systematic comprehensive analysis of the PSO and including the diverse applications of the algorithm [6]. AlRashidi et al. presented a comprehensive coverage of different PSO applications in solving optimization problems in the area of electric power systems. and discusseed PSO possible future applications in the area of electric power systems and its potential theoretical studies [7]. Russell C.Eberhart and Yuhui Shi analyzed the engineering and computer science aspects of developments, applications, and resources related to PSO [66]. Mr. Ninad K. Kulkarni et al. reviewed the applications of PSO in mechanical domain and also described its improved version [9]. Ekrem özge and Bekir Aksoy analyzed the trajectory planning of the robotic arm using the PSO [10]. Ahmed G. Gad systematically reviewed the PSO, including algorithm improvements, diverse application domains, open issues and future perspectives [11]. Tareq M. Shami et al provided a comprehensive review of PSO including remarkable engineering applications [12]. Tiwari Sukriti and Ashwani Kumar provided insights on basic notions and progress of the PSO in power system applications [13]. Janmenjoy Nayak,et al. presented an in-depth analysis of PSO with its developments from 1995 to 2020 [63]. John A. Ramirez-Figueroa et al proposed a new method for disjoint principal component analysis based on PSO [15]. Yu Xue et al successfully applied PSO to optimize the topological construction of the echo state networks [16]. Pace Francesca et al reviewed the application of the PSO to perform stochastic inverse modeling of geophysical data[17]. Ali R. Kashani et al provided a detailed review of applications of PSO on different geotechnical problems [18]. There are also many related books that involve theory and application [20, 21, 22, 23, 24, 25]. PSO has proven to be extremely useful in various disciplines.
On the other hand, there is also a lot of work on improving the PSO. Mainly, PSO has been modified by following strategies: Dynamic adjustment strategy of inertia weight; Asynchronous optimization design of learning factors; Algorithm fusion and multi-strategy collaborative improvement; Speed and location update mechanism innovation; Parameter adaptation and diversity enhancement, and so on.
Since the introduction of the PSO algorithm in 1995 by Kennedy and Eberhart [1, 2], the PSO algorithm has attracted a great attention
modification of the PSO controlling parameters, hybridizing PSO with other well-known meta-heuristic algorithms such as genetic algorithm (GA) and differential evolution (DE), cooperation and multi-swarm techniques. This paper attempts to provide a comprehensive review of PSO, including the basic concepts of PSO, binary PSO, neighborhood topologies in PSO, recent and historical PSO variants, remarkable engineering applications of PSO, and its drawbacks.
Moreover, this paper reviews recent studies that utilize PSO to solve feature selection problems. Finally, eight potential research directions that can help researchers further enhance the performance of PSO are provided.
Davoud Sedighizadeh, et al introduced the Generalized Particle Swarm Optimization algorithm as a new version of the PSO algorithm for continuous space optimization [19].
2 Particle Swarm Optimization
Particle swarm optimization(PSO) is a swarm intelligent algorithm that simulates the foraging process of animals in nature, and it is a evolutionary algorithm[58, 59, 60, 26]. Every candidate solution is called particle in (1). All candidate solutions are called swarm. Each particle is characterized by two factors, i.e., position and velocity , where and denote the ith particle and its dimension in search space . The fitness of each particle can be evaluated according to the objective function in (1). PSO starts with the random initialization of a swarm in the search space . Then, every particle update its velocity and position based on its own experience and the experience of optimal particle. represents the optimal position of the -th particle within iteration steps. denotes the best position in the swarm so far. The updating rule of velocity and position is shown in (2) and (3).
[TABLE]
where is the swarm size. and represent the velocity and position of particle at -th iteration step respectively. These mainly are addition and subtraction of vectors, such as and . and are two independent random numbers between [math] and . and are acceleration constants, usually , which determine the relative pull of and ; is called the inertia weight factor which can be chosen by a random number. Generally, the value of each component in can be clamped to the range . It controls excessive roaming of particles outside the searching space. Each particle updates its position according to Eqs.(2) and (3). In this way, all particles find their new positions and apply these new positions to update their individual best positions and global best position of the swarm. This process continues until the user specified stopping criterion is met, such as maximum iteration number, the optimal solution tends to stabilize or total error requirement. The particle is drawn towards and . For more details see references[58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70] and the references cited therein.
In the PSO, each particle calculates its fitness value and compares it with the fitness values of its global neighbors (i.e. the entire particle swarm) to obtain the global optimal value (social influence). At the same time, each particle calculates its own historical fitness value and compares it with its current fitness value to obtain the optimal fitness value of the individual, that is, the individual extremum (learning from experience).
A vector is a quantity that has two independent properties: magnitude and direction. Specially, a free vector can move freely in parallel. Both velocity and position are free vectors in . For example, the addition of vectors follows the parallelogram rule (or equivalent to the triangle rule ) in , which is shown in Fig.1(a). The meaning of , and of equation (2) are shown in Fig.1(b). Food (apple) represents the globally optimal solution. In the previous iterations, represents the current globally optimal solution. , , , is the trajectory of the -th particle. is the individual optimal solution of the -th particle. and typically possess significant influence over . Velocity is influenced by three parts of information in the equation(2). is inertia of velocity. represents individual cognition part. is social part. This means that individual shares the optimal location information of the swarm.
Equations (2) and (3) constitute the main structure of PSO. The speed of the next generation is influenced by the speed of their previous generation , its own previous generation position , the individual extremum () of its previous generation and the global extremum () of the previous generation of the swarm. These three variables(, and ) have the greatest impact on it, and retaining only these three variables simplifies the particle swarm algorithm and can also achieve good convergence results. In fact, is not only influenced by these three quantities. The position (, ) of other individuals can also have an impact on it. Although the impact is weak, we can still not ignore these information. How to embed the position information of all other particles (, ) into each iteration process (Eqs.(2) and (3))? And how to ensure that the position and velocity information of the -th particle is influenced by all other particles?How to design such a new particle swarm algorithm is the problem that this article needs to consider. Based on a series of three experiments, show that in most cases this new particle swarm algorithm will have better optimization performance than the PSO.
The particle swarm optimization algorithm can be understood as a spatiotemporal system. and represent space and time, respectively. Therefore, can it be connected to another famous spatiotemporal system - the coupled map lattice system? The answer is affirmative.
The important way to improve particle swarm optimization is to modify its neighborhood topology structure [36, 27, 28, 29, 30, 31, 32, 33, 34, 35], These topological structures include but are not limited to the following types: Small-world network, ring topology, fully connected topology, star topology, mesh Topology, tree topology, toroidal topology, flustering algorithms, dynamic neighborhood topology, and so on.
For example, the fully informed PSO adopts the following velocity update formula, while the position update formula remains unchanged [36, 27, 28, 29].
[TABLE]
where known as a constriction factor. is the velocity vector for individual . is a uniform random number generator. is the number of neighbors particle has and is neighbor. If includes only itself and its best neighbor, then this formula degenerates to the PSO. is the best solution in the neighbourhood of the particle .
In [38, 39, 40, 41], clustering algorithm is integrated into PSO. For example, the speed update formula is as follows [41].
[TABLE]
where represents the number of subgroups in the clustering process. means the optimal solution in the -th subgroup during the -th iteration process. During each iteration, the particle swarm is divided into several subgroups using clustering methods, and the optimal solution in each subgroup is integrated into the PSO. The performance of particle swarm optimization based on clustering method is superior to that of PSO.
In [37], the speed update formula is as follows.
[TABLE]
where is sarm size. In the first iterations, represents the current optimal solution value of the -th dimensional component of the -th particle. During each iteration, the local optimal solutions of all particles participate in the velocity update.
3 Coupled Map Lattice
A coupled map lattice(CML) is a high-dimensional discrete dynamical system formed by a given discrete dynamical system (mapping) interacting (coupling) in a certain way. It was first proposed by Kuniko Kaneko[45].
The first simple CML is proposed for studying spatiotemporal chaos: considering the phenomena generated by local chaotic processes and spatial diffusion processes. Take one-dimensional mapping as the simplest representative chaos and use discrete Laplace operator for diffusion. Some phenomena in nature can be represented by reaction-diffusion equations, namely
[TABLE]
The local reaction process can be represented by a nonlinear mapping , for example, Logistic mapping . And the diffusion process is obtained by the discrete Laplace operator
[TABLE]
Combine the above local reaction processes with the diffusion process represented by the discrete Laplace operator to obtain a coupled map lattice
[TABLE]
represents discrete time. is system state. is lattice coordinates (). is system size. is coupling parameter (coupling strength), and is a local function or lattice function[45, 46]. Usually, periodic boundary conditions are taken, i.e.,
Coupled map lattice with different types of topological structures have been proposed. The selection of coupling topology structure will affect the dynamics of the entire system. Coupled map lattice is relatively simple, but it has most of the characteristics of spatiotemporal chaos [45, 46, 47, 48, 49, 51, 52, 53, 54]. Usually, the calculations for each lattice point are exactly the same, and some results of the theory of low dimensional dynamical systems can still be applied to their analysis. Coupled map lattices and their applications have been widely studied. Fore example, some common topological structures of couple map lattices are as follows.
Globally coupled map lattice [48, 49, 50]
[TABLE]
The information at time of the -th lattice point is partially influenced by the information of all lattice points at time , and can be understood as weight coefficient.
Accumulated coupled map lattice [47]
[TABLE]
The information at time of the -th lattice point is partially influenced by the information of the first lattice points and itself at time . It is the accumulation of information from the previous lattice points.
There are many forms of coupled map lattice systems, such as cross coupled map lattice systems, two-dimensional coupled map lattice systems, etc. In coupled map lattice systems, the adopted lattice functions are usually one-dimensional chaotic maps, such as logistic maps, Tent maps, etc. Therefore, coupled map lattice systems have complex dynamic properties and it has a wide rang of applications, especially in the field of chaotic image encryption. The mutual influence of information in coupled map lattice systems inspires us to apply this idea to particle swarm optimization algorithms and integrate it with the foraging process of birds. We link the information exchange process of birds to a coupled map lattice systems.
4 Globally Coupled Particle Swarm Optimization
Particle swarm optimization algorithm is a method of information sharing among the birds, and continuously updating information of the birds by searching for the optimal bird’s position information. Coupled map lattice system is the mutual influence between lattice points, which is somewhat similar to the mutual influence between birds. Therefore, we attempt to integrate the coupled map lattice with the particle swarm optimization algorithm to propose a new modified particle swarm optimization algorithm. Now we will embed the concept of globally coupled map lattice into particle swarm optimization algorithm and . A new particle swarm optimization algorithm will be proposed. We name it particle swarm optimization with coupled map lattice (denoted as PSOCML).
Two update iteration formulas of PSOCML are as shown in equations and , where the velocity update formula undergoes significant changes. The coupling principle of the globally coupled map lattice is utilized here.
[TABLE]
where is the swarm size, and represent the velocity and position of particle at iteration step respectively. is the coupling strength. indicates that information of the global optimal position affects the current particle , and describes that the global optimal position has an impact on other particles. is social part. Compared to particle swarm optimization, it contains much information of swarm. Denote , and , the schematic diagram of equation (12) is shown in Fig.2. These also demonstrate that everything is interconnected, and each bird has an influence on other birds, and it is also influenced by others. Although the form of (12) is more complex than equation in particle swarm optimization algorithm, the principle is still simple and effective. This algorithm will have better ability to find the optimal value. represents all other particles () are drawn towards . These information also affects the flight speed of particle . () is the weight value of information, usually has a smaller value.
The pseudocode PSOCML is explained in Algorithm.
Remark 1*.*
When , the equations and degenerate into PSO.
Remark 2*.*
When , the equation becomes
[TABLE]
This situation (14) will be analyzed in the future. Whether it is superior to PSO is still unknown.
Remark 3*.*
Coupled map lattice exhibits spatiotemporal chaos [45, 46, 47, 48, 49, 51, 52, 53, 54], and its orbit has ergodicity. When the time is long enough, the orbit will fill the entire phase space. These chaotic features are suitable for updating particle communities in PSO [42]. kanta Matsumoto and Chihiro Ikuta proposed the new PSO with CML and worker ant’s law [43]. The CML is applied to the moving equation of PSO[43, 44], and the modification of the velocity update formula is as follows.
[TABLE]
Where is the amplitude controller of output of CML. is chaotic orbits of CML. is independent from , and each particle . Adding this term () has disturbance effect on the velocity value to avoid the algorithm getting stuck in local optima too early. Here CML is added to the velocity equation of each particle. CML helps PSO escaping out from the local solutions. CML enhances the search capability of the PSO. This method involves chaotic perturbation of velocity. This increases ergodicity of speed, thereby affecting the positional information of particles. This is different from various topological structures of PSO [36, 27, 28, 29, 30, 31, 32, 33, 34, 35].
5 Experiments and Results
The comparison is made with the PSO, and other variations of PSO. PSO and most of its improved algorithms are designed to solve single objective global optimal problems. These can not find multiple optimal points of a multimodal function. Therefore, the ones we have selected below are all single objective testing functions.
The performance of the GCPSO is tested on modern benchmark functions, i.e., CEC 2017 (Awad et al., 2016). The performance of the GCPSO is compared with PSO and other improved PSO.
In the second test suite (), CEC2017 test suite (Awad et al., 2016) (F2 has been excluded because it shows unstable behavior especially for higher dimensions, and significant performance variations for the same algorithm implemented in Matlab) are selected to evaluate the performance the proposed GCPSO. Originally CEC consisted of functions. Then, function number 2 was excluded from the competition. At the time of writing this note (March 2021), an updated problem definitions renumbered functions formerly used in (old function is now called , etc.). These benchmark functions are classified into four categories: unimodal functions (), multimodal functions (), hybrid functions (), composition functions (). These functions are used to further verify the validity of the proposed GCPSO. The problem dimension, population size and max number of fitness evaluations are set uniformly as 30, 40 and . All the PSO algorithms are run for independent times. These shifted and rotated functions are more complex and make our test results more convincing.
6 Conclusion
We integrate the framework of the globally coupled map lattice system into the particle swarm optimization algorithm and propose a new set of globally coupled particle swarm optimization algorithms. After experimental testing, the new globally coupled particle swarm optimization has excellent optimization ability.
Acknowledgements
This work is supported by (No. ).
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Kennedy J, Eberhart R. Particle swarm optimization[C]//Proceedings of ICNN’95-international conference on neural networks. ieee, 1995, 4: 1942-1948.
- 2[2] Eberhart R, Kennedy J. A new optimizer using particle swarm theory[C]//MHS’95. Proceedings of the sixth international symposium on micro machine and human science. Ieee, 1995: 39-43.
- 3[3] Charilogis V, Tsoulos IG, Tzallas A. An improved parallel particle swarm optimization. SN Computer Science. 2023, 4(6):766.
- 4[4] Plevris V, Solorzano G. A Collection of 30 Multidimensional Functions for Global Optimization Benchmarking. Data. 2022; 7(4):46.
- 5[5] Anula Khare, Saroj Rangnekar. A review of particle swarm optimization and its applications in solar photovoltaic system. Applied Soft Computing 13.5 (2013): 2997-3006.
- 6[6] Houssein, E. H., Gad, A. G., Hussain, K., & \& Suganthan, P. N. (2021). Major advances in particle swarm optimization: theory, analysis, and application. Swarm and Evolutionary Computation, 63, 100868.
- 7[7] M. R. Al Rashidi, M. E. El-Hawary, A Survey of Particle Swarm Optimization Applications in Electric Power Systems, in IEEE Transactions on Evolutionary Computation, vol. 13, no. 4, pp. 913-918, Aug. 2009.
- 8[8] Shi, Yuhui. Particle swarm optimization: developments, applications and resources. Proceedings of the 2001 congress on evolutionary computation (IEEE Cat. No. 01TH 8546). Vol. 1. IEEE, 2001.
