An Improved Red-Billed Blue Magpie Optimization Algorithm for 3D UAV Path Planning in Complex Terrain

TL;DR

This paper introduces an improved optimization algorithm for 3D drone path planning in complex environments, enhancing performance and efficiency.

Contribution

The CTWRBMO algorithm introduces four enhancement strategies to improve global search and avoid local optima in 3D UAV path planning.

Findings

CTWRBMO outperforms existing algorithms in optimization accuracy and convergence speed.

Paths planned with CTWRBMO are shorter, safer, and smoother in complex terrain simulations.

The algorithm effectively balances exploration and exploitation for practical UAV applications.

Abstract

This paper presents the Circle-Mapping Transition and Weighted Red-Billed Blue Magpie Optimizer (CTWRBMO), designed to address significant challenges in 3D path planning for drones. Although the original Red-Billed Blue Magpie Optimizer (RBMO) algorithm features a simple structure, few parameters, and strong local search capability, making it well-suited for UAV path optimization, it suffers from insufficient population diversity, limited global search ability, and a tendency to fall into local optima in complex high-dimensional scenarios. To overcome these limitations, four enhancement strategies are introduced. Firstly, the Circle chaotic mapping strategy leverages the randomness and ergodicity of chaotic sequences to generate an initial population that is uniformly distributed. This enhancement improves population diversity from the beginning and provides a solid foundation for global optimization. Secondly, the ε parameter is dynamically adjusted to prioritize local refinement during the early stages of optimization. This adjustment enables rapid convergence toward potentially optimal areas. This parameter increases to enhance global search capabilities as the algorithm progresses, thereby broadening the optimization space and achieving a dynamic equilibrium. Additionally, a nonlinear dynamic weighting factor (wd) is incorporated into the position update formula. The algorithm’s ability to escape local optima is significantly improved by dynamically altering the weight ratio between historical optimal positions and the current position. Furthermore, an elite perturbation mechanism based on individual neighborhoods is implemented to generate candidate solutions using local information. This mechanism enhances the algorithm’s local exploration capabilities and improves the stability of preserving optimal solutions, supported by a greedy criterion for optimal retention. Experimental results show that the CTWRBMO algorithm significantly outperforms comparison algorithms in terms of optimization accuracy and convergence speed, demonstrating exceptional global optimization capabilities. Additional applications in UAV 3D path planning simulations evaluated paths based on length, threat avoidance efficiency, and smoothness. The results indicate that paths planned using CTWRBMO are shorter, safer, and smoother compared to those generated by the Harrier Hawks Optimization (HHO), African Vulture Optimization Algorithm (AVOA), Artificial Bee Colony (ABC) Algorithm, and the traditional Magpie Algorithm, effectively meeting practical engineering requirements for UAV 3D path planning.

Figures

Full text

1. Introduction

Uncrewed aerial vehicles (UAVs) have rapidly evolved into an essential component of general aviation, showcasing applications in precision agriculture, logistics delivery, emergency response, disaster warning, aerial photography, and more. Two critical technologies that facilitate autonomous drone flight are path planning and obstacle avoidance. Any static or dynamic object can act as an obstacle during a drone’s flight, potentially causing damage to the aircraft or even leading to a crash in a three-dimensional environment. In addition to external obstacles, UAVs face constraints related to their own flight characteristics during path planning, which include flight altitude, turn angles, and maximum and minimum flight distances. Therefore, effective UAV path planning must not only focus on avoiding obstacles but also consider these inherent limitations of the vehicle. These factors significantly influence the efficiency and safety of mission execution. Path planning approaches for UAVs can generally be categorized into two main groups: classical algorithms and swarm-intelligence-based optimization techniques. Classical algorithms have been widely used for route computation, such as A* [1,2], Dijkstra’s algorithm [3], and genetic algorithms [4]. However, these methods often require considerable computational resources and memory when deployed in large-scale or high-dimensional environments. Researchers have begun to explore swarm intelligence optimization algorithms, including the Sparrow Search Algorithm [5], the Grey Wolf Optimizer [6], and the Artificial Bee Colony [7]. These algorithms mimic natural behaviors, such as foraging and mating, demonstrating strong global search capabilities and adaptability. Reference [8] integrated a bidirectional search strategy to enhance pheromone updating and next-node selection, effectively reducing the risk of premature convergence. Reference [9] incorporated genetic operations into Particle Swarm Optimization (PSO) to accelerate population convergence speed. Reference [10] utilized chaotic mapping during initialization and introduced dynamic boundary reverse learning to improve solution diversity and prevent local stagnation. Reference [11] introduced the Cauchy distribution for perturbation and adopted the greedy strategy to retain the better solutions, enhancing the escape ability, accelerating the convergence speed in the later stage, and improving the convergence accuracy of the algorithm. Reference [12] adopted an improved reverse learning strategy to expand the search space, enhancing the algorithm’s exploration ability for high-quality solutions, avoiding missing potential optimal paths. In Reference [13], the African vulture optimization algorithm integrates the discovery and vigilance mechanisms of the sparrow algorithm to balance exploration and exploitation, improving the algorithm’s early global search ability. Other metaheuristic algorithms, such as the Whale Optimization Algorithm [14], Dung Beetle Optimization [15], and Butterfly Optimization Algorithm [16], have also been applied to UAV path planning.

RBMO [17] is a recently developed swarm intelligence algorithm inspired by the cooperative foraging, predation, and food-hoarding behaviors of red-billed blue magpies. Benefiting from its cooperative population-based search mechanism, RBMO facilitates effective information sharing among individuals and has demonstrated promising performance in continuous optimization problems. Such characteristics are particularly relevant to three-dimensional UAV path planning, where the optimization variables are continuous, and the objective landscape is highly nonlinear due to terrain variation and obstacle distribution.

Nevertheless, the canonical RBMO exhibits several limitations when extended to strongly constrained and high-dimensional optimization scenarios. In particular, limited population diversity during the early search stage may reduce exploration capability, while the absence of adaptive regulation in the search dynamics can result in premature convergence and insufficient global exploration. In complex UAV path planning environments involving terrain constraints, obstacles, no-fly zones, and vehicle kinematics, these shortcomings may lead to infeasible trajectories or suboptimal flight paths.

Despite these limitations, RBMO was deliberately selected as the foundation of this study due to its flexible algorithmic structure, explicit behavioral mechanisms, and established effectiveness in continuous optimization. Compared with more rigid optimization frameworks, RBMO provides a suitable baseline for systematic enhancement, allowing targeted modifications to be integrated while preserving its inherent cooperative search advantages. This makes RBMO a meaningful candidate for adaptation to multi-constraint three-dimensional UAV path planning.

To overcome the aforementioned deficiencies, this study extends RBMO by incorporating several complementary strategies aimed at enhancing population diversity, strengthening global exploration, and improving convergence stability under complex constraints. The primary objective is to retain the intrinsic strengths of RBMO while significantly improving its robustness and solution quality in challenging and constrained UAV path planning environments.

2. Red-Billed Blue Magpie Optimization Algorithm

The RBMO algorithm is a metaheuristic approach inspired by the natural behaviors of red-billed blue magpies, including foraging, pursuing, striking, and storing food. Its optimization mechanism can be conceptually divided into three sequential stages: prey exploration, prey capture, and food preservation.

2.1. Prey Searching

Red-billed blue magpies typically search for food either in small groups (2 to 5 individuals) or large flocks (more than 10 individuals). To simulate their search strategies under different group sizes, Equation (1) is used to model the small-group searching behavior, while Equation (2) represents the large-flock searching behavior.

$$[eqn]$$ $$[eqn]$$

t denotes the current iteration number; p represents a randomly selected subgroup of 2–5 red-billed blue magpies from the entire population, while q denotes a group consisting of more than 10 individuals randomly chosen from the same population. The symbol $[eqn]$ refers to the position of the m-th red-billed blue magpie randomly selected at iteration t, and $[eqn]$ represents the position of a randomly chosen individual in the current iteration.

2.2. Prey Attacking

The red-billed blue magpies primarily target small prey or plants during the prey-attacking phase when acting in small groups. This behavioral pattern can be represented by Equation (4). When foraging in large flocks, the magpies tend to attack larger prey such as big insects or small vertebrates, which is described by Equation (5).

$$[eqn]$$ $$[eqn]$$ $$[eqn]$$

Here, $[eqn]$ denotes the current global best position, while $[eqn]$ and $[eqn]$ represent normally distributed random numbers with a mean of 0 and a standard deviation of 1.

2.3. Food Storage

The Red-billed Blue Magpie also stores excess food in hidden places such as tree holes for future use in addition to searching for food and attacking prey. During this process, information about potential solutions is preserved, which assists individuals in finding the global optimal solution, as expressed in Equation (6).

$$[eqn]$$

The terms $[eqn]$ and $[eqn]$ represent the fitness values of the $[eqn]$ -th Red-billed Blue Magpie before and after the position update, respectively.

The following pseudocode (Algorithm 1) illustrates the detailed procedure. Algorithm 1 The RBMO 1:Begin2:Initialize the relevant parameters (T, N, ε, etc.)3:     while t < T4:            Calculate the fitness of each search agent5:            Update the best solution6:            **Exploration:**7:                 if rand < ε8:                         Update the position of red-billed blue magpie by Equation (1)9:                 else10:                          Update the position of red-billed blue magpie by Equation (2)11:                 end if12:             **Exploitation:**13:                 if rand < ε14:                          Update the position of red-billed blue magpie by Equation (4)15:                 Else16:                          Update the position of red-billed blue magpie by Equation (5)17:                 end if18:             $[eqn]$ and accomplish food storage by Equation (6)19:          end while20:          return best solution21:end

3. Improved Red-Billed Blue Magpie Optimization Algorithm

3.1. Circle Chaotic Mapping

The initial positions of the population are generated randomly in the RBMO algorithm, which cannot cover the entire search space effectively and results in suboptimal performance. The Circle chaotic mapping method is introduced for population initialization to address this limitation. Chaotic mapping exhibits excellent ergodicity and uniformity, ensuring that the population is more evenly distributed in the search space and achieves better optimization performance than random initialization. The mathematical expression of the Circle chaotic map is given in Equation (7):

$$[eqn]$$

where $[eqn]$ denotes the current value of the chaotic sequence at the $[eqn]$ -th iteration. Parameter $[eqn]$ is the modulation parameter, which affects the traversal speed and uniformity of the sequence. When $[eqn]$ , the sequence is prone to local contraction, reducing the coverage ability of the search space; when $[eqn]$ , the chaotic characteristics of the system weaken, which is not conducive to maintaining the randomness of the sequence. Research [18,19,20] indicates that $[eqn]$ can achieve a better balance between chaos and stability, so we adopted this value. Parameter $[eqn]$ is the nonlinear strength coefficient, used to adjust the amplitude and nonlinearity of the chaotic sequence. A smaller $[eqn]$ value will weaken the chaos of the system, while an excessively large k value may cause the sequence to enter a periodic zone or diverge. Based on the literature and the preliminary experimental results of this paper, $[eqn]$ can maintain a large chaotic domain and enhance the diversity of the initial solution.

As shown in Figure 1, the left subfigure (a) represents the probability of each value, while the right subfigure (b) displays the chaotic points generated by the Circle chaotic mapping. It can be observed that using the Circle chaotic mapping to construct the initial population of the Red-billed Blue Magpie results in a more uniform distribution across the search space, higher population diversity, and an expanded search range for the algorithm.

3.2. Dynamic Adjustment of ε

The parameter ε serves as a key factor controlling whether individuals act in small groups or as a swarm when searching for food or attacking prey in the RBMO algorithm. The individuals tend to form small groups, focusing on local exploitation when the random number is less than ε. The individuals are more likely to act as a swarm, enhancing global exploration and helping the algorithm escape from local optima when the random number is greater than ε. The algorithm may suffer from insufficient exploration in the early stages and inadequate exploitation in the later stages if a fixed $[eqn]$ value is used. Therefore, dynamically adjusting $[eqn]$ helps avoid this issue by strengthening global search capability during the early phase and gradually converging toward local optimization in the later phase, thereby improving the overall solution quality of the algorithm.

The expression for $[eqn]$ is given in Equation (8).

$$[eqn]$$

where $[eqn]$ represents the maximum number of iterations, and $[eqn]$ denotes the current iteration number. Parameter $[eqn]$ is the initial coefficient of the dynamic adjustment $[eqn]$ , which controls the global exploration intensity of the CTWRBMO algorithm in its early stage. To determine an appropriate initial value, we conducted a sensitivity analysis on parameter $[eqn]$ using the CEC2020 benchmark function F2. For each candidate value, the algorithm was independently executed 30 times, and the convergence behavior was recorded. The results (Figure 2) show that $[eqn]$ approximately equal to 0.62 provides the best balance between exploration and exploitation, yielding stable convergence and superior average performance., $[eqn]$ = 0.62 is selected as the initial coefficient of $[eqn]$ in CTWRBMO.

3.3. Nonlinear Dynamic Weight Factor wd

The disturbance amplitude during the position updating process significantly affects the search performance in the (RBMO) algorithm. The nonlinear dynamic weight factor $[eqn]$ , based on an exponential function, is introduced to achieve a more effective balance between global exploration and local exploitation. The value of $[eqn]$ is close to 1 during the early iterations, maintaining a strong disturbance intensity that encourages individuals to perform large-scale jumps within the solution space, which promotes global search and helps avoid local optima. The value of $[eqn]$ gradually decreases toward 0 in the later stages, limiting the disturbance amplitude to enhance local exploitation and accelerate convergence.

The expression of $[eqn]$ is given in Equation (9).

$$[eqn]$$

where $[eqn]$ represents the maximum number of iterations, and $[eqn]$ denotes the current iteration number.

3.4. Elite Perturbation and Elite Retention Strategy Based on Individual Neighborhood

All individuals use the current global best solution $[eqn]$ as the guiding reference in the original (RBMO) algorithm, causing the entire population to converge toward it. Although this approach accelerates convergence, it easily leads the population to cluster in a single region and fall into local optima. Therefore, instead of directly using the global best position $[eqn]$ to guide the population, each individual generates a new locally perturbed candidate solution $[eqn]$ based on its neighborhood information. Then, following a greedy criterion, the algorithm determines whether to replace the current individual’s position. This mechanism prevents the population from converging excessively toward the best global position and improves search diversity and local exploitation capability.

During the foraging phase, Red-Billed Blue Magpies search for food either in groups or clusters. When cooperating in groups, each individual generates a new solution based on the positional differences of selected members, as defined in Equation (10):

$$[eqn]$$

When the search is performed in a clustered form, the update rule is given as follows:

$$[eqn]$$

Both strategies generate perturbed solutions by constructing neighborhood-based difference vectors centered on the current individual $[eqn]$ , rather than being guided by the global best solution $[eqn]$ .

After each perturbation update, the algorithm applies a greedy criterion to determine whether to accept the newly generated solution. If the fitness of $[eqn]$ is better than that of the original position $[eqn]$ , the new position replaces the old one; otherwise, the previous solution is retained, as shown in Equation (12):

$$[eqn]$$

During the prey-attacking phase, the same local perturbation mechanism is used to generate new solutions, and a coefficient $[eqn]$ is introduced to refine the perturbation of candidate solutions, as expressed in Equations (13) and (14):

$$[eqn]$$ $$[eqn]$$

All the above update operators are formulated in terms of three-dimensional position vectors $[eqn]$ . The movement direction of each individual in the three-dimensional search space is determined by neighborhood difference vectors, while the displacement magnitude is jointly regulated by random perturbation terms and dynamic weighting factors. In this way, the search process can achieve a smooth transition from large-scale global exploration to refined local exploitation within a three-dimensional environment.

During the update process, a newly generated three-dimensional position is accepted only if it leads to an improvement in fitness; that is, a greedy selection strategy is adopted to ensure that the path solutions continuously converge toward better optima in the three-dimensional search space.

The algorithm applies a greedy criterion to determine whether to accept the newly generated solution after each perturbation update. In addition, the algorithm records the current best individual $[eqn]$ in each iteration and outputs it as the final solution, ensuring the optimality and stability of the search results.

To further improve the interpretability of the elite disturbance mechanism, the overall workflow of this mechanism is illustrated in Figure 3. The flowchart summarizes the individual neighborhood-based perturbation strategies corresponding to Equations (10), (11), (13) and (14), clearly showing that each candidate solution is generated by perturbing the individual position $[eqn]$ , rather than being directly guided by the global best solution $[eqn]$ .

The following pseudocode (Algorithm 2) represents the detailed algorithmic procedure: Algorithm 2 The CTWRBMO1:Begin2: $[eqn]$ , $[eqn]$ , etc.) 3:Initialize population using Circle chaotic map (Equation (7))4:Evaluate initial fitness of all individuals5: $[eqn]$ ← best initial solution6:     while t < T7:Calculate the fitness of each search agent8:                 $[eqn]$ if improved9:                Dynamically update ε by (8)10:                Dynamically updated wd by (9)11:                **Exploration:**12:                     if rand < ε13:                             Generate candidate $[eqn]$ using Equation (10) 14:                     Else15:                             Generate candidate $[eqn]$ using Equation (11) 16:                     end if17:                **Exploitation:**18:                     if rand < ε19:                             Update red-billed blue magpie position using Equation (13)20:                     Else21:                             Update red-billed blue magpie position using Equation (14)22:                     end if23:                     Apply greedy selection by Equation (12)24:                      $[eqn]$ if new solution is better25:     end while26:     Return global best solution $[eqn]$ 27:end

3.5. Comparison Between Original RBMO and CTWRBMO

To further highlight the improvement strategies proposed in this paper, a systematic comparison between the original RBMO and CTWRBMO is conducted at both the mathematical structure and algorithmic mechanism levels. As shown in Table 1, the differences between the two in key aspects such as initialization methods, step size adjustment, search updates, and elite retention are summarized.

The original RBMO employs uniform random initialization, which, although simple, results in substantial randomness in the population’s initial spatial coverage. This issue becomes more pronounced when the search space is large or the terrain is highly variable, making it difficult to ensure adequate initial solution quality. To address this limitation, CTWRBMO replaces random initialization with the Circle chaotic map, whose strong ergodicity and uniformity enable more thorough coverage of the search space during the initialization stage. This enhancement improves the initial population quality and reduces the sensitivity of convergence speed to initial distribution.

During the search phase, RBMO uses a fixed search probability ε, leading to an invariant balance between exploration and exploitation throughout the iteration process. While such a setting may yield stable performance on simple functions, it becomes inadequate in complex constrained environments, where a fixed ε cannot simultaneously support early-stage global exploration and late-stage local refinement. In contrast, CTWRBMO adopts a linearly decreasing ε(t), which starts at a relatively large value to strengthen global search in the early iterations and gradually decreases to facilitate fine-grained exploitation later on. This mechanism provides a more adaptive balance between exploration and exploitation.

The binary limitation of RBMO is also reflected in its step-size control. The disturbance amplitude changes only slightly during iterations, making it difficult for the algorithm to achieve both fast convergence and sufficient refinement accuracy. CTWRBMO introduces a nonlinear decaying weight factor, wd, to regulate the step size through exponential attenuation. This not only helps expand the search range in the early iterations but also enables finer adjustments near high-quality solutions as the search converges.

Furthermore, the position update in RBMO relies primarily on the global best individual. As iterations progress, such single-directional guidance tends to make the population converge prematurely, reducing diversity and increasing the risk of being trapped in local optima. To mitigate this, CTWRBMO incorporates neighborhood-based differential perturbation and employs a greedy selection mechanism. By integrating local neighborhood candidates while preserving elite solutions, CTWRBMO maintains necessary perturbations during the search, prevents premature convergence, and enhances search stability.

Overall, CTWRBMO introduces targeted improvements to initialization, exploration–exploitation regulation, and position update mechanisms, enabling it to achieve superior convergence stability and optimization accuracy in complex terrain environments.

3.6. Algorithm Time Complexity Analysis

Time complexity is an important indicator for evaluating algorithm performance. Let the problem dimension be $[eqn]$ , and let the computational complexity of a single objective function evaluation be $[eqn]$ . The initialization stage of CTWRBMO includes chaotic mapping and fitness evaluation, with a complexity of $[eqn]$ . During the position update stage, each individual performs two update operations, each requiring the computation of the mean value over a subset of size $[eqn]$ . In the worst case, the update complexity is $[eqn]$ . Meanwhile, each iteration involves $[eqn]$ objective function evaluations, resulting in a complexity of $[eqn]$ .

The chaotic initialization, dynamic weighting factor, and exponential convergence factor introduce only constant-level or $[eqn]$ -level computational overhead and therefore do not change the overall order of complexity. Consequently, the overall time complexity of CTWRBMO is $[eqn]$ , which is of the same order as that of the original RBMO algorithm.

4. Algorithm Performance Evaluation

4.1. Algorithm Parameter Settings

This study selects eight representative algorithms to comparatively evaluate the performance of the proposed CTWRBMO, including the classical benchmark algorithms PSO [21] and Grey Wolf Optimizer (GWO) [22]; recently developed advanced swarm intelligence optimization algorithms HHO [23], Whale Optimization Algorithm (WOA) [24], Gorilla Troops Optimizer (GTO) [25], and AVOA [26]; the state-of-the-art algorithm L-SHADE [27]; and RBMO as a baseline algorithm. These comparisons are conducted to validate the practical effectiveness of CTWRBMO and to assess the impact of the proposed improvements on algorithmic performance. The parameter settings of all algorithms are summarized in Table 2.

4.2. Performance Comparison of Algorithms

Systematic tests and assessments were conducted based on the CEC2020 benchmark function set (detailed in Table 3) to evaluate the performance of the improved RBMO algorithm, comparing CTWRBMO with multiple algorithms, including PSO, AVOA, WOA, GTO, GWO, L-SHADE, and RBMO. Each benchmark function was independently executed 30 times to rigorously examine the convergence characteristics and optimization capabilities of CTWRBMO, and the experimental data are presented in Table 4. To ensure fair comparison and reliability of the experimental results, all algorithms involved in the analysis were uniformly configured with a population size of $[eqn]$ , a maximum of $[eqn]$ iterations, and a problem dimension of $[eqn]$ , minimizing the influence of the inherent stochasticity of heuristic algorithms on performance evaluation. The performance of each algorithm was comprehensively analyzed and compared using measures such as the best value, worst value, and mean value. All algorithms were implemented and simulated using MATLAB R2022a on a platform equipped with an AMD Ryzen 7 8845H processor (Advanced Micro Devices, Inc., Santa Clara, CA, USA), with a base clock speed of 3.80 GHz.

The CEC2020 benchmark suite consists of 10 single-objective test functions, categorized into unimodal functions (F1), basic functions (F2–F4), composition functions (F5–F7), and hybrid functions (F8–F10), where $[eqn]$ denotes the optimal value of the $[eqn]$ -th test function.

The experimental results of different algorithms on the CEC2020 benchmark functions are presented in Table 4. Overall, the proposed CTWRBMO algorithm demonstrates superior convergence performance and stronger robustness on most test functions. Specifically, on functions F1, F4, and F6, the optimal values obtained by CTWRBMO are very close to the theoretical optima; meanwhile, on functions F3, F5, F7, F8, F9, and F10, its best solutions consistently outperform those of all comparison algorithms, indicating that the proposed algorithm achieves higher convergence accuracy on these problems.

The mean values of the algorithms reflect their overall optimization performance over 30 independent runs. As shown in Table 4, CTWRBMO achieves the best average performance on F1, F2, F3, F4, F7, F8, and F9, demonstrating its ability to stably obtain high-quality solutions across repeated experiments. In addition, for most test functions, CTWRBMO yields significantly smaller standard deviations than the other algorithms, indicating lower performance variability and stronger robustness. These statistical results suggest that CTWRBMO is able to maintain stable convergence behavior and reliable solution quality across different runs.

Figure 4 presents the average convergence curves of the nine algorithms on the test functions. These results are based on the mean performance obtained from 30 independent runs of each algorithm and provide an intuitive comparison of convergence speed and accuracy. As can be observed, CTWRBMO exhibits a more pronounced downward trend in the early iterations, indicating a faster convergence speed at the initial stage. On F1, F2, F3, F4, F7, F8, and F9, the convergence curves of CTWRBMO consistently lie below those of all comparison algorithms, demonstrating higher final solution accuracy.

It should be noted that on the test functions F6 and F10, the final convergence accuracy of CTWRBMO is slightly inferior to that of L-SHADE, and on F5, it is slightly inferior to RBMO, indicating that its performance advantage is not absolute on all test functions. However, the differences in the final optimal values among the algorithms on these functions are relatively small, and CTWRBMO still maintains a faster overall convergence speed, suggesting that it remains highly competitive for complex optimization problems.

Overall, CTWRBMO demonstrates faster convergence, higher solution accuracy, and better stability on most test functions, while its performance advantage is relatively weakened on a few functions, indicating that the algorithm still exhibits certain performance variations under different problem characteristics.

4.3. Wilcoxon Rank-Sum Test

In this study, the Wilcoxon rank-sum test was employed to statistically analyze the best results obtained from 30 independent runs, with the aim of evaluating the performance differences between the proposed CTWRBMO algorithm and the comparative algorithms. A significance level of p < 0.05 was adopted to indicate statistically significant differences between two algorithms. The statistical results are summarized in Table 5. For most benchmark functions from F1 to F10, the p-values between CTWRBMO and the other algorithms are lower than 0.05, demonstrating that the performance improvement of CTWRBMO is statistically significant. These results indicate that CTWRBMO exhibits significant advantages in terms of optimization accuracy and stability. Overall, CTWRBMO significantly outperforms the other algorithms on the majority of the test functions, and the observed performance improvements are supported by clear statistical evidence, thereby further validating the effectiveness and robustness of the proposed method.

4.4. Ablation Experiments

To verify the effectiveness of the proposed strategies, ablation experiments were conducted in this study. Four different algorithm variants were constructed by incorporating each strategy individually into the RBMO framework. Specifically, IRBMO1 incorporates only the Circle chaotic mapping, IRBMO2 incorporates only the nonlinear dynamic weighting factor $[eqn]$ , IRBMO3 incorporates only the $[eqn]$ parameter, and IRBMO4 incorporates only the elite perturbation and elite retention strategy based on individual neighborhoods.

Six representative benchmark functions from CEC2020 (F2, F3, F4, F7, F9, and F10) were selected for performance evaluation. In all experimental settings, algorithmic parameters were strictly kept identical. Each algorithm was independently executed 30 times, with a maximum of 500 iterations per run, in order to reduce the influence of randomness. To intuitively compare the contribution of each strategy, the symbols “+/=/−” were used to summarize the relative performance of each algorithm compared with the original RBMO in terms of average optimization results. Specifically, “+” indicates the number of cases in which the algorithm outperforms RBMO, “=” denotes equivalent performance, and “−” indicates inferior performance relative to RBMO.

As shown in Table 6, the “+/=/−” statistics of IRBMO1 are 17/1/0, indicating that IRBMO1 achieves superior average convergence accuracy compared with RBMO on most benchmark test functions. Since RBMO itself can reach the theoretical optimum on function F4, IRBMO1 also converges to this theoretical optimum, resulting in one “=”. This result demonstrates that circular chaotic initialization can effectively enhance the initial quality of the population, allowing the search process to start from more promising initial positions and thereby improving the overall convergence performance of the algorithm.

The “+/=/−” value of IRBMO2 is likewise 17/1/0, and it can be observed from Figure 5 that IRBMO2 converges significantly faster than RBMO. This is because, during the early stages of iteration, the nonlinear weighting factor maintains a relatively large value, which enlarges the search step size and enables the population to rapidly move toward regions with high-quality solutions, thereby markedly accelerating the overall convergence speed.

The “+/=/−” statistics of IRBMO3 are also 17/1/0. Although IRBMO3 exhibits faster convergence on some functions (e.g., F7 and F9) and slightly slower convergence than RBMO on the remaining functions, it achieves higher final average convergence accuracy, with standard deviations consistently lower than those of RBMO. This indicates that the dynamic ε strategy can effectively enhance both the stability of the algorithm and the quality of the obtained solutions, and thus has considerable value as an improvement strategy.

The “+/=/−” value of IRBMO4 is 17/1/0, and its convergence accuracy is likewise superior to that of RBMO. As observed in Figure 5, IRBMO4 is able to escape from local optima multiple times. This is mainly attributed to the introduction of the elite disturbance mechanism, which generates perturbed candidate solutions based on the current individual position $[eqn]$ and its neighborhood information, and then determines whether to accept the new solutions according to a greedy criterion. This mechanism prevents the population from prematurely concentrating in a single optimal region while simultaneously enhancing local search accuracy, thereby improving population diversity and the ability to escape local optima when solving complex optimization functions.

5. Problem Formulation

5.1. Problem Description

UAV path planning aims to find an optimal flight trajectory from the start point to the target point while satisfying the UAV’s operational constraints, such as maximum and minimum flight distance, turning angles, and threat zones. The planned path must not only account for external environmental factors, including terrain variations and threat areas, but also adhere to the UAV’s fundamental performance limitations, encompassing maximum range, minimum segment length, maximum turning angle, and allowable speed range.

5.2. 3D UAV Flight Environment Modeling

Before executing a mission, the UAV must first model the environment to simulate its actual flight conditions. In constructing the environment model, this study assumes static terrain, fixed obstacles, and no wind interference, thereby considering only the UAV’s flight characteristics in a static environment. The integrated smoothing algorithm proposed in [28] satisfies the UAV’s longitudinal maneuverability constraints while closely fitting the original terrain and slightly elevating it, resulting in a safer and more reliable path planning process. In this study, the integrated smoothing algorithm was employed to model the terrain, resulting in an equivalent terrain surface $[eqn]$ . To ensure that the UAV maintains the necessary vertical safety clearance throughout the flight, a minimum safe altitude h above the equivalent terrain is added to account for potential collision risks due to terrain variations. This yields a modified terrain surface $[eqn]$ , as illustrated in Figure 6, which is expressed as follows:

$$[eqn]$$

5.3. Constraint Conditions

5.3.1. Maximum Flight Distance Constraint

UAVs are subject to range constraints during flight due to limited energy resources (battery or fuel), and the flight path must respect the maximum allowable distance. The maximum safe flight distance is assumed to be $[eqn]$ to ensure mission completion and avoid energy depletion.

The total flight distance must satisfy the following constraints:

$$[eqn]$$

Here, $[eqn]$ denotes the number of flight segments, and $[eqn]$ represents the flight distance of the $[eqn]$ -th segment.

5.3.2. Minimum Flight Distance Constraint

The flight distance $[eqn]$ , between any two consecutive flight segments is not excessively short to avoid frequent directional changes during flight, which increase energy consumption, the drone must ensure that. Assuming a minimum flight distance of $[eqn]$ , each segment’s distance must satisfy:

$$[eqn]$$

5.3.3. Maximum Turning Angle Constraint

At each path waypoint, a UAV must adjust its heading based on the directional change between the preceding and succeeding flight segments. If the turning angle is too large, the UAV may be unable to generate sufficient centripetal force through attitude adjustment to complete the maneuver, potentially leading to attitude oscillations, path deviations, or even stall. Therefore, it is necessary to set an upper limit, $[eqn]$ , on the maximum turning angle between consecutive flight segments to ensure trajectory feasibility and flight stability. For a sequence of path waypoints $[eqn]$ , the flight direction vectors before and after waypoint $[eqn]$ are given by $[eqn]$ and $[eqn]$ , respectively, and the UAV’s turning angle at waypoint $[eqn]$ , $[eqn]$ , is calculated as the angle between these two vectors:

$$[eqn]$$

To avoid sharp turns and ensure a feasible flight, the turning angle must satisfy the following constraint:

$$[eqn]$$

Here, $[eqn]$ denotes the maximum safe turning angle that the UAV can achieve while remaining within the allowable limits of its attitude, lateral acceleration, and structural load-bearing capacity.

5.3.4. Maximum Climb/Descent Rate Constraints

When the altitude difference between consecutive path nodes is too large, the UAV may not be able to adjust its height within the available time, leading to safety risks such as insufficient thrust due to excessive climbing or aerodynamic instability caused by rapid descent. Therefore, it is necessary to impose a maximum climb rate $[eqn]$ and a maximum descent rate $[eqn]$ during path planning to ensure that the trajectory remains vertically feasible. Let the altitudes of consecutive path nodes be $[eqn]$ and $[eqn]$ , and let the flight time interval be $[eqn]$ ; the vertical velocity along this segment is given by

$$[eqn]$$

The vertical velocity of the UAV is required to satisfy the following constraint:

$$[eqn]$$

where $[eqn]$ denotes the UAV’s average vertical velocity along the $[eqn]$ -th flight segment, ensuring that the altitude change remains within a controllable range.

5.3.5. Flight Speed Constraints

The flight speed of a UAV is generally influenced by factors such as the propulsion system thrust, aerodynamic drag, endurance, and structural limitations of the aircraft. Flying at too low a speed may pose a risk of stall, while excessively high speeds may result in structural overload or unstable maneuvering. Therefore, it is necessary to impose speed limits for each segment in the planned path.

Let the spatial distance between two consecutive nodes $[eqn]$ and $[eqn]$ be $[eqn]$ , and $[eqn]$ denote the assumed time interval between these nodes. The average speed along this segment can then be expressed as follows:

$$[eqn]$$

The flight speed should satisfy the following constraint:

$$[eqn]$$

where $[eqn]$ and $[eqn]$ represent the allowable minimum and maximum safe flight speeds, respectively, ensuring that the UAV maintains controllability and stability throughout the entire flight path.

5.3.6. Collision Avoidance Constraint

Unmanned aerial vehicles (UAVs) may encounter various obstacles such as mountains, hills, and no-fly zones during path planning. This paper employs spherical objects to simulate obstacles, as illustrated in Figure 7a. Let a threat area be centered at $[eqn]$ with radius $[eqn]$ ; the interior of this sphere is strictly prohibited for UAV entry. Suppose a UAV is expected to travel from node $[eqn]$ to node $[eqn]$ . Although $[eqn]$ and $[eqn]$ themselves are located outside the threat region, the flight path between them may intersect the spherical threat area centered at $[eqn]$ with radius $[eqn]$ (see Figure 7b). To prevent the UAV from intruding into this restricted zone, collision avoidance measures must be applied. In this study, additional path nodes are dynamically inserted between $[eqn]$ and $[eqn]$ to enable the UAV to smoothly bypass the threat region and avoid potential hazards.

To achieve this, the path segments are refined using a uniform node distribution approach. Let the straight-line distance from the start point to the end point be $[eqn]$ , and let the desired spacing between adjacent path nodes be $[eqn]$ . The number of additional nodes required along the path can then be calculated as

$$[eqn]$$

By refining the path nodes, the UAV can perform more precise collision detection with threat areas during the path optimization process, ensuring that the generated flight path does not enter spherical threat zones and thereby satisfying the collision avoidance constraints.

5.4. Cost Function

In the process of unmanned aerial vehicle (UAV) path planning, the cost function serves as the criterion for evaluating the relative merits of different paths. A well-designed cost function guides the algorithm to generate safe and efficient paths within complex environments, avoiding terrain obstacles and hazardous zones. This paper constructs the following UAV path cost function by comprehensively considering multiple factors, including maximum flight range, minimum flight altitude, turn angle, and avoidance of hazardous zones:

$$[eqn]$$

The coefficients ω_1_, ω_2_, ω_3_, and ω_4_ represent the weights assigned to distance, altitude, turning angle, and threat costs, respectively. Following the principles of safety and operational efficiency, these weight coefficients are set as follows:

Threat weight ω_4_ = 5: The threat cost is the most critical safety constraint in path planning. Entering a threat zone may result in mission failure; therefore, the threat cost is assigned the highest weight to ensure that the algorithm always prioritizes path safety.

Distance weight ω_1_ = 1: Path length directly affects UAV energy consumption and mission duration. As a key factor secondary only to safety, it is assigned a moderately high weight to encourage the algorithm to search for shorter paths.

Altitude weight ω_3_ = 0.5: The turning angle cost mainly influences path smoothness and flight maneuverability. Its importance is lower than that of threat avoidance and path length, so it is assigned a moderate weight to prevent unnecessary sharp turns while avoiding excessive penalization of path curvature.

Angle weight ω_2_ = 0.05: The magnitude of the altitude cost is typically much higher than other cost terms. Without proper normalization, it could dominate the optimization process. Therefore, a relatively small weight is assigned to ensure safe altitude while minimizing its interference with distance and threat optimization.

5.4.1. Distance Cost

The path distance directly affects the UAV’s energy consumption and flight time; therefore, it is essential to consider the distance between each consecutive path node during path planning. The distance cost term, $[eqn]$ , can be obtained by summing the Euclidean distances of all consecutive segments along the path, which is calculated as follows:

$$[eqn]$$

Here, $[eqn]$ represents the spatial distance between two consecutive nodes $[eqn]$ and $[eqn]$ , while $[eqn]$ denotes the spatial distance between the consecutive nodes $[eqn]$ and $[eqn]$ .

5.4.2. Angular Cost

In UAV path planning, the smoothness of path turning plays a critical role in ensuring flight stability and reducing maneuvering load. Excessively large turning angles can not only increase lateral overload but also lead to abrupt maneuvers, higher energy consumption, and in some cases, infeasible flight actions. Therefore, in this study, an angular cost term, $[eqn]$ , is introduced into the objective function to penalize large turning angles along the path, thereby improving the overall path smoothness. The angular cost term is calculated as follows:

$$[eqn]$$

5.4.3. Altitude Cost

During the UAV’s flight from the start point to the endpoint, both terrain clearance and altitude constraints must be satisfied. To ensure flight safety, the UAV should neither fly too low, which would increase the risk of collision, nor fly excessively high, which would result in unnecessary energy consumption. To address this, an altitude cost term is incorporated into the objective function to penalize unsafe altitude behaviors. If the flight path maintains a safe altitude and changes smoothly, the corresponding cost remains low. Conversely, if the flight altitude falls below the minimum allowable height $[eqn]$ or approaches the terrain height $[eqn]$ , the cost increases significantly to avoid potential collision risks. Terrain elevation data are obtained from a pre-established three-dimensional digital elevation model (DEM). For any given path point $[eqn]$ , the corresponding terrain height $[eqn]$ is determined through terrain interpolation and contributes to the computation of the altitude cost $[eqn]$ , as expressed in Equation (28).

$$[eqn]$$

Here, $[eqn]$ denotes the flight altitude at the $[eqn]$ -th path point, $[eqn]$ represents the minimum allowable flight altitude, and $[eqn]$ indicates the terrain elevation at the $[eqn]$ -th point. The parameter $[eqn]$ defines the minimum safety distance to avoid collision with the terrain. $[eqn]$ is the penalty coefficient applied when the flight altitude falls below the minimum allowable altitude $[eqn]$ , while $[eqn]$ is the penalty coefficient applied when the flight altitude is lower than the sum of the terrain height and the safety distance. These penalties are used to strictly prevent the UAV from flying through the terrain.

5.4.4. Threat Costs

In practical flight environments, UAVs must strictly avoid obstacles, terrain, and no-fly zones. Therefore, this study incorporates a threat evaluation function $[eqn]$ into the cost function to determine whether any point $[eqn]$ along the path approaches or enters the $[eqn]$ -th threat region.

$$[eqn]$$

Each threat source is modeled as a spherical region, where $[eqn]$ represents the center of the threat and $[eqn]$ denotes its radius. An additional safety buffer of $[eqn]$ is applied around each sphere to mitigate the risk of boundary crossing due to positioning errors or airflow disturbances. When $[eqn]$ , it indicates that the path intersects with an obstacle or a no-fly zone, and the path is immediately deemed infeasible. In this study, all threat sources are treated as forbidden regions; if any of these regions are violated, a penalty is imposed on the total cost function, as defined in Equation (30):

$$[eqn]$$

Here, $[eqn]$ denotes the total number of threat sources. If $[eqn]$ , it indicates that the path has entered an obstacle or a no-fly zone at least once, and consequently, $[eqn]$ is forcibly increased by approximately tenfold. Conversely, if $[eqn]$ , all path nodes are within safe regions, and a smaller penalty is applied to encourage maintaining a safe distance.

5.5. UAV 3D Path Planning Simulation and Analysis

In this work, three-dimensional UAV path planning experiments were performed using MATLAB R2022a. To enable the UAV to effectively circumvent terrain obstacles and reach the destination safely, the planning framework was developed on a 3D terrain model incorporating predefined threat regions. Two mountainous environments were constructed, containing three and six threat zones, respectively, to emulate flight scenarios of differing complexity and evaluate the CTWRBMO algorithm’s performance under various environmental conditions. The UAV’s operational domain was specified as a $[eqn]$ spatial grid. The initial position was set to (15.15, 30.3, 295.9), while the destination point was defined as (449.5, 459.6, 422). A summary of the environmental configurations is presented in Table 7, and detailed parameters of the threat models are listed in Table 8. The corresponding 3D terrain model generated from these parameters is depicted in Figure 8.

In this study, three-dimensional UAV path planning in Environments 1 and 2 was investigated using five optimization algorithms—ABC, AVOA, HHO, RBMO, and the proposed CTWRBMO. The selection of these algorithms is based mainly on two considerations. First, ABC is a classical benchmark algorithm that provides a reliable baseline for comparison; HHO and AVOA are recently proposed algorithms that have demonstrated competitive performance in recent years; and RBMO is adopted as the baseline algorithm for improvement to verify the effectiveness of the proposed strategies. In addition, during preliminary exploratory experiments, we observed that in strongly constrained three-dimensional environments involving complex terrain, obstacles, and multiple kinematic constraints, general-purpose swarm intelligence algorithms such as PSO, GTO, and WOA have difficulty in stably generating feasible and safe paths when no additional constraint-handling or path-repair mechanisms are incorporated. To avoid compromising the fairness and consistency of the comparative study due to the use of different constraint-handling strategies across algorithms, these algorithms were not included in the three-dimensional path planning comparisons.

Furthermore, in order to enhance the ability of the algorithm to generate feasible solutions in a strongly constrained three-dimensional environment and to further improve the comparability among different algorithms, this study introduced the same initial path generation strategy for all the compared algorithms to construct feasible initial solutions. This initial path serves only as a starting reference for the search process and does not participate in the fitness evaluation during the subsequent path optimization process, thereby avoiding any bias impact on the final optimization result.

To further guarantee the comparability of the experimental results, the population size of all algorithms was set to 50, and the maximum number of iterations was fixed at 100. Each algorithm was independently executed 30 times in both Environment 1 and Environment 2. The optimal, mean, and variance of the path lengths obtained from these 30 independent runs are reported. A smaller mean indicates a lower average path planning cost, while the variance reflects the stability and robustness of the algorithms in three-dimensional environments.

In addition, multiple performance metrics were evaluated, including average computational time, path feasibility rate, path smoothness, and energy consumption, to comprehensively assess the performance and applicability of each algorithm. The energy consumption of each UAV path was calculated using a simplified model that considers the Euclidean distance between consecutive waypoints, the path curvature (which indirectly reflects the UAV’s maximum turning angle constraint), and vertical energy expenditure (which indirectly reflects climb and descent rates). Although simplified, this model provides a consistent and fair basis for comparing different algorithms. The parameter settings for all algorithms are summarized in Table 9.

This study presents representative three-dimensional side views and top views of typical experimental results to facilitate an intuitive comparison of the performance of different algorithms in three-dimensional UAV path planning. The results for Environment 1 are illustrated in Figure 9a,b. In this scenario, the number of threat regions is relatively small. The path planned by the CTWRBMO algorithm exhibits overall altitude stability, with a smooth flight trajectory throughout the mission and no abrupt turning angles, demonstrating strong obstacle avoidance capability. In contrast, the UAV paths generated by the other algorithms show larger altitude fluctuations and tend to fly closer to mountainous terrain, indicating weaker obstacle avoidance and threat avoidance capabilities, which makes flight safety difficult to guarantee.

The results for Environment 2 are shown in Figure 9c, where a larger number of threat regions are present. Except for the CTWRBMO algorithm, the paths generated by the other algorithms exhibit significant altitude fluctuations and poor trajectory smoothness. Nevertheless, even in this more complex environment, the path produced by CTWRBMO maintains minimal altitude variation and superior smoothness, as further illustrated by the top view in Figure 9d. The paths generated by RBMO and ABC contain large turning angles, which increase the overall path length and result in sharper maneuvers. By comparison, the CTWRBMO algorithm produces a smoother and shorter path.

Table 10 summarizes the path planning performance of the different algorithms. In Environment 1, the average path cost of CTWRBMO is lower than that of RBMO, HHO, ABC, and AVOA by 6.04%, 36.23%, 4.29%, and 31.43%, respectively. In Environment 2, the reductions are 6.29%, 30.22%, 6.18%, and 31.38%, respectively, indicating that CTWRBMO is highly effective in minimizing path length.

Path success rate was used to evaluate the reliability of an algorithm in generating feasible paths. The results show that both CTWRBMO and ABC achieve a 100% success rate in both environments, meaning that all paths generated during 30 independent runs were feasible and did not pass through threat zones or produce abnormal trajectories. RBMO maintains a 100% success rate in Environment 1 but drops to 93% in Environment 2, suggesting reduced robustness under higher threat density. HHO and AVOA exhibit lower success rates, often generating paths that violate no-fly zones or contain excessive sharp turns.

Regarding path smoothness, CTWRBMO produces the smallest average turning angles, indicating that its trajectories are smoother compared to those generated by the other algorithms.

Energy consumption and average computational time were also evaluated. CTWRBMO consistently achieves the lowest energy consumption due to its shorter and smoother paths, which minimize frequent sharp-turn operations. In contrast, the other algorithms consume more energy because their paths are longer and less smooth, requiring more maneuvers. The average computational time of CTWRBMO is comparable to RBMO and slightly higher than ABC or AVOA, demonstrating that the improved solution quality does not incur a prohibitive computational cost.

It is noteworthy that CTWRBMO exhibits the smallest variation in path cost across independent runs, indicating minimal performance fluctuation and high stability. Overall, these results demonstrate that CTWRBMO can reliably generate high-quality, feasible, smooth, and energy-efficient paths while maintaining acceptable computational times, making it particularly suitable for complex 3D UAV path planning scenarios.

From the convergence curves of the different algorithms in Environment 1 (Figure 10a), it can be observed that during the first 25 iterations, CTWRBMO ranks third in convergence speed, slightly behind AVOA and ABC. However, as the iterations progress, by the 60th iteration, the convergence accuracy of CTWRBMO surpasses that of both AVOA and ABC, achieving the best overall performance among all algorithms. This indicates that, although CTWRBMO explores the solution space slightly more slowly in the early stage, its optimization strategies enable efficient solution refinement in subsequent iterations, ultimately achieving higher convergence accuracy.

Similarly, in Environment 2 (Figure 10b), CTWRBMO exhibits relatively slower convergence in the early iterations. Nevertheless, around the 45th iteration, its convergence accuracy exceeds that of ABC, rising to the top rank. These results demonstrate that CTWRBMO can effectively generate high-quality solutions in the later stages of iteration, highlighting its robustness and excellent global search capability across different 3D UAV path planning scenarios.

6. Conclusions

This paper presents an enhanced version of the Red-billed Blue-jay Optimization Algorithm (CTWRBMO), which integrates multiple strategies to improve the accuracy and computational efficiency of three-dimensional path planning. The introduction of Circle chaotic mapping during the population initialization phase enhances the diversity of initial solutions, effectively mitigating the risk of being trapped in local optima. In addition, the algorithm incorporates a dynamic ε adjustment mechanism and a nonlinear dynamic weight factor to balance global search and regional exploration, enabling extensive exploration and comprehensive exploitation of high-quality solutions. Furthermore, a local search strategy based on elite perturbation is proposed to enhance the stability and overall balance of the search process, effectively addressing the premature convergence issues commonly observed in the traditional RBMO algorithm.

Simulation results demonstrate that CTWRBMO consistently achieves the lowest average path cost across different environments with varying numbers of threat targets. The algorithm also exhibits the smallest variance in path cost, indicating high adaptability and stability in response to changes in initial conditions and environmental uncertainties. In terms of convergence speed and solution accuracy, CTWRBMO performs excellently, generating smooth, safe, and reliable optimal paths in complex and dynamic flight scenarios, as evidenced by convergence curve analyses.

The performance of CTWRBMO carries important practical implications for real UAV systems. By generating shorter and smoother paths, the algorithm effectively reduces energy consumption and extends flight endurance, which is particularly critical for UAV platforms with limited battery capacity. Its consistently high path feasibility and low variance ensure safe navigation in environments containing obstacles, threat zones, or complex terrain, thereby enhancing mission reliability. These characteristics highlight the algorithm’s capability to balance solution quality, operational safety, and computational efficiency. Although CTWRBMO demonstrates strong performance under the current experimental conditions, its applicability to multi-objective optimization tasks and real-time path planning requires further investigation. Future research will involve in-depth testing in actual flight environments to comprehensively evaluate its engineering feasibility and robustness in practical applications.