Reference Point and Grid Method-Based Evolutionary Algorithm with Entropy for Many-Objective Optimization Problems

TL;DR

This paper introduces RGEA, a new algorithm that combines reference points and grids with entropy to better solve complex many-objective optimization problems.

Contribution

The novel RGEA algorithm uses entropy to adaptively switch between reference point and grid methods for improved performance on many-objective problems.

Findings

RGEA outperforms existing algorithms on both regular and irregular Pareto front problems.

Entropy helps RGEA determine whether to use reference points or grids based on the problem's shape.

Extensive experiments show RGEA's effectiveness across 3-to-10 objective problems.

Abstract

In everyday scenarios, there are many challenges involving multi-objective optimization. As the count of objective functions rises to four or beyond, the problem’s complexity intensifies considerably, often making it challenging for traditional algorithms to arrive at satisfactory solutions. The non-dominated sorting evolutionary reference point-based (NSGA-III) and the grid-based evolutionary algorithms (GrEA) are two prevalent algorithms for many-objective optimization. These two algorithms preserve population diversity by employing reference point and grid mechanisms, respectively. However, they still have limitations when addressing many-objective optimization problems. Due to the uniform distribution of reference points, the reference point-based methods do not obtain good performance on problems with an irregular Pareto front, while grid-based methods do not achieve good results on problems with a regular Pareto front because of the uneven partition of grids. To address the limitations of reference point-based algorithms and grid-based approaches in tackling both regular and irregular problems, a reference point- and grid-based evolutionary algorithm with entropy is proposed for many-objective optimization, denoted as RGEA, which aims to solve both regular and irregular many-objective optimization problems. Entropy is introduced to measure the shape of the Pareto front of a many-objective optimization problem. In RGEA, a parameter α is introduced to determine the interval for calculating the entropy value. By comparing the current entropy value with the maximum entropy value, the reference point-based method or the grid-based method can be determined. In order to verify the performance of the proposed algorithm, a comprehensive experiment was designed on some popular test suites with 3-to-10 objectives. In addition, RGEA was compared against six algorithms without adaptive technology and six algorithms with adaptive technology. A great number of experimental results were obtained showing that RGEA can obtain good results.

Figures

Tables

Table 1. Characteristics of the test problems.

Test Problems	Objectives (M)	Variables (D)	Evaluations	Pareto Front
DTLZ1	3, 5, 8, 10	M-1 + 5	50,000	Regular
DTLZ2, DTLZ4	3, 5, 8, 10	M-1 + 10	20,000	Regular
DTLZ3	3, 5, 8, 10	M-1 + 10	50,000	Regular
DTLZ5, DTLZ6	3, 5, 8, 10	M-1 + 10	20,000	Irregular
DTLZ7	3, 5, 8, 10	M-1 + 20	20,000	Irregular
WFG1-3	3, 5, 8, 10	24	N∗500	Irregular
WFG4-9	3, 5, 8, 10	24	N∗200	Regular

Full text

1. Introduction

Multi-objective optimization problems (MOPs) are ubiquitous in production and life, such as industry and scientific research. The primary objective of MOPs is to identify a set of equilibrium solutions, known as Pareto optimal solutions. When the number of optimization objectives exceeds three, they are termed many-objective optimization problems (MaOPs) [1]. Traditional multi-objective optimization algorithms cannot meet the needs of many-objective optimization. MaOPs pose some new challenges to existing multi-objective evolutionary algorithms [2,3]. First, Pareto dominance becomes degraded for many-objective optimization problems. In the population, there are many non-dominated solutions, which leads to losing selection pressure. Secondly, the recombination operator may be inefficient. Finally, it becomes difficult and expensive to maintain and evaluate the diversity of solutions in the population. Hence, numerous algorithms have been devised for solving these challenges, which can be divided into five categories.

Multi-objective optimization algorithm based on Pareto dominance: On the one hand, this type of algorithm modifies the Pareto dominance domain of the solution, either by expanding or reducing it, thereby enhancing convergence by refining the Pareto dominance mechanism. When handling with many-objective optimization problems, multi-objective optimization algorithms based on Pareto dominance produce a large number of non-dominance solutions in the population, leading to poor pressure of dominance. The representative algorithms are $[eqn]$ -dominance [4], grid dominance [5], fuzzy dominance [6], $[eqn]$ -dominance [7], etc. On the other hand, for the purpose of ensuring a better selection of the Pareto front, some Pareto dominance-based algorithms have adopted other mechanisms, among which the popular algorithms are the $[eqn]$ dominance-based evolutionary algorithm ( $[eqn]$ -DEA) [8], vector angle-based evolutionary algorithm (VaEA) [9], shift-based density estimation algorithm (SDE) [10], knee point-driven evolutionary algorithm (KnEA) [11], etc.

Multi-objective optimization algorithm based on indicators: This kind of algorithms use performance indicators—such as the binary indicators $[eqn]$ , R2, and HV—as fitness measures to direct the evolutionary process. The classical algorithms include the indicator-based evolutionary algorithm (IBEA) [12], improved metaheuristic evolutionary multi-objective algorithm based on the R2 indicator (MOMBI-II) [13], and the S metric selection evolutionary multi-objective algorithm (SMS-EMOA) [14]. In order to obtain high-quality solutions, SRA [15] achieves two objectives and mixing based on random bubbling. The indicator-based multi-objective optimization algorithms have good theoretical properties, but they require expensive computation time.

Multi-objective optimization algorithm based on objective dimension reduction: This kind of algorithm forms a minimum conflict objective set by removing redundant objectives so as to simplify the scale of the problem. The widely used algorithms include $[eqn]$ -MOSS [16], NL-MVU-PCA [17], and nonlinear correlation information entropy [18]. According to the similar characteristics of objective dimension reduction and feature selection in machine learning, ref. [19] applied feature selection technology to objective dimension reduction, transforming the objective dimension reduction problem into a multi-objective optimization problem, and they then solved this by an evolutionary algorithm. The algorithm could lose important information when the number of objectives are reduced.

Multi-objective optimization algorithm based on decomposition: This kind of algorithm uses aggregation functions, including the weight sum method (WS) [20], the Tchebycheff method (TCH) [21], and the penalty-based boundary intersection method (PBI) [22], to transform a multi-objective optimization problem into a number of single-objective optimization sub-problems, and it also uses an evolutionary algorithm to optimize these sub-problems at the same time. However, due to the difficulty of maintaining diversity in the traditional decomposition-based multi-objective optimization algorithm (MOEA/D) [23], some improved MOEA/D algorithms have been proposed. The typical algorithms include I-DBEA [24], MOEA/D-DU [25], EFR-RR [25], MOEA/DD [26], MOEA/D-LWS [27], etc.

Multi-objective optimization algorithm based on reference points: This class of algorithm creates a uniformly distributed set of reference points in the objective space to partition it, linking each solution in the population to its nearest reference point. These reference points serve as search directions, guiding the evolutionary process. This class of algorithm shows excellent performance on many-objective optimization problems with a regular Pareto front, but, on the irregular problems, they see a degradation in performance. These sort of algorithms include NSGAIII [2], RVEA [28], crEA [29], etc.

In recent years, multi-objective optimization algorithms that incorporate reference points and grid strategies have garnered significant attention due to their outstanding performance in addressing MaOPs. The representative algorithms are the reference point-based non-dominated sorting evolutionary algorithm (NSGA-III) and the grid-based evolutionary algorithm (GrEA). NSGA-III employs a set of widely distributed reference points to ensure population diversity, and it implements non-dominated sorting. On the other hand, GrEA assesses individuals based on grid domination and grid difference criteria. However, NSGA-III shows excellent performance for regular many-objective optimization problems, but it falls short when handling irregular ones. Conversely, GrEA excels in dealing with irregular many-objective optimization problems but does not perform well with regular ones. In addition, researchers have proposed a large number of methods for dealing with irregular problems; however, the reference point-based method [30,31,32] and the clustering method, or grid-based method [33,34], are still mainly used. There is no reasonable mechanism designed to integrate the reference point-based method and the grid-based method. Hence, a key issue is how to combine the two methods to handle irregular and regular many-objective optimization problems. Entropy has been widely used in the many-objective evolutionary algorithm [35,36,37]. In [38], entropy was treated as a diversity metric to measure the diversity performance of the algorithm. In [39], entropy was researched as a termination criterion for many-objective evolutionary algorithms. However, the entropy was computed based on grid technology. In [40], the entropy was computed based on a reference point and could be viewed as a criterion whether the Pareto front of the many-objective optimization problem was regular or irregular. As such, the entropy mechanism could be employed to combine the two methods. By comparing the current entropy value with the maximum entropy value, the reference point-based method or the grid-based method can be determined.

Drawing from the preceding discussion, this paper introduces a novel algorithm that combines the reference point-based method and the grid-based method for solving regular and irregular many-objective optimization problems (RGEA). RGEA aims to address the shortcomings of NSGA-III and GrEA via a new criterion with entropy, combining the reference point-based method and the grid-based method to improve the diversity of solutions, enhance search efficiency, and ensure the uniform distribution of solutions on the Pareto front. Experiments are conducted on the DTLZ and WFG test problems to verify the effectiveness of RGEA. The proposed algorithm RGEA was compared with six advanced algorithms without adaptive technology, such as MOEA/D, MOEA/D-DE, NSGAIII, GrEA, MSOPS-II, and IBEA, and six advanced algorithms with adaptive technology, such as I-DBEA, ANSGAIII, MOEA/D-AWA, RVEAa, DEAGNG, and AdaW. The primary novel contributions of this paper can be summarized as outlined below.

A new algorithm was developed to combine reference point-based and grid-based methods in coping with regular and irregular many-objective optimization problems. The algorithm proposed is to remedy the issue where the reference point-based method has good results in dealing with regular problems but fails to obtain good solutions in dealing with irregular problems, as well as address how the grid-based method has good results in dealing with irregular problems but does not deliver good solutions for regular problems.In this paper, an entropy-based criterion, which combines the reference point-based and grid-based methods, is proposed. In order to determine the interval of the calculated entropy value, we introduced a parameter $[eqn]$ . By comparing the current entropy with the maximum entropy, we can determine whether to adopt the reference-point-based approach or the grid-based approach.A comprehensive experimental evaluation was conducted to assess the performance of RGEA. To verify its effectiveness, this paper, using regular and irregular DTLZ1-7 and WFG1-9 test suites for verification, compared RGEA with six evolutionary algorithms without adaptive technology and six evolutionary algorithms with adaptive technology. The large number of experimental results obtained indicate that the proposed algorithm RGEA can achieve good performance on many-objective optimization problems with a regular and irregular Pareto front.

The structure of the subsequent chapters in this paper is organized as follows: Section 2 delves into the definitions of multi-objective optimization problems, differentiating between regular and irregular issues. In addition, it reviews pertinent research concerning reference point-based algorithms and grid-based approaches for many-objective optimization. Section 3 presents the strategy and the overall framework of the proposed algorithm. The experimental setup is detailed in Section 4, and this is followed by Section 5, which discusses the experimental results obtained from various test suites. Lastly, Section 6 provides a concise summary of the paper.

2. Preliminaries

2.1. Definition of the Multi-Objective Optimization Problem

A MOP can be defined as

$$[eqn]$$ $$[eqn]$$

where D denotes the decision space of the problem, and $[eqn]$ is the objective space of a problem, which is composed of m objective functions $[eqn]$ . When the number of objectives exceeds three, it is named a MaOP [1]. Let $[eqn]$ and $[eqn]$ , then $[eqn]$ is said to dominate $[eqn]$ if $[eqn]$ for all $[eqn]$ and $[eqn]$ . A point $[eqn]$ is named Pareto-optimal if there is no $[eqn]$ such that $[eqn]$ dominates $[eqn]$ . The set of all the Pareto optimal points is named the Pareto set (PS). $[eqn]$ is named the Pareto front (PF) [39], $[eqn]$ is named the ideal point if $[eqn]$ is the minimal value of $[eqn]$ over the decision space, and $[eqn]$ is called the nadir point if $[eqn]$ is the maximal value of of $[eqn]$ over the PS.

2.2. Definition of Regular and Irregular Problems

A regular multi-objective optimization problem (MOP) is characterized by a Pareto front (PF) that resembles the shape of a simplex. Specifically, when the ideal point of such a problem is considered as the origin, all of the vectors in the positive direction intersect with this ideal point [41]. Conversely, any MOP with PF shapes deviating from this pattern is classified as an irregular MOP. The specific description is to let $[eqn]$ be a vector, which is subject to $[eqn]$ :

$$[eqn]$$ $$[eqn]$$

where Formula (2) represents that, for the problem with regular PF, there is no v that does not intersect with the PF of the problem, that is, all of the vectors in the positive direction intersect with the PF of the problem. Formula (3) indicates that, for a problem with an irregular PF, there must be a v that does not intersect with the PF of the problem.

2.3. Reference Point-Based Many-Objective Optimization Algorithms

Moen H. J. F. et al. [42] introduced a novel method for creating a uniformly distributed set of reference points utilizing the Manhattan distance as the basis. This approach involves generating the subsequent generation of solutions by comparing the solutions associated with the same reference points based on this distance metric. K. Deb and Himanshu Jai [2] introduced a concept that combines reference points with a non-dominated sorting. Initially, individuals in the first $[eqn]$ layer are chosen through non-dominated sorting. Subsequently, the remaining individuals in the final l layer are selected based on two criteria: the distance between the solution and its associated reference point, as well as the number of solutions associated with the same reference point. Yuan Y., Xu H., and Wang B. et al. [8] employed a $[eqn]$ -dominance criterion to allocate solutions to distinct clusters, each represented by specific reference points. Within each cluster, individuals compete against one another. Cheng R. et al. [28] introduced a novel mechanism for partitioning the objective space into multiple subspaces by means of reference vectors. Within each of these subspaces, selection operations are conducted independently. Essentially, this division operation imposes a constraint on the subproblem associated with each reference vector, thereby demonstrating its effectiveness in balancing convergence and diversity within decomposition-based methods [43]. Cai L. et al. [29] suggested utilizing clustering operators to preserve the diversity among solutions. Additionally, they proposed constructing a structured set of reference points to generate a collection of uniformly distributed units within the objective space. Seada H. and Deb K. [44] developed a unified optimization algorithm, which is an extension of NSGAIII, to tackle various multi-objective optimization problems. This algorithm employs a population size larger than the number of reference vectors and incorporates a selection operator. Y. Xiang et al. [45] introduced a novel multi-objective evolutionary algorithm utilizing an adaptive Pareto reference point approach. Within this framework, the choice of reference points is contingent upon the shape of the Pareto optimal front (PF), enabling the calculation of convergence and diversity indices for individuals. W. Yu et al. [46] introduced a decomposition algorithm that leverages an adaptive reference vector Q-learning method. Within this framework, the reinforcement Q-learning framework treats the adaptive process of reference vectors as a reinforcement learning challenge, enabling these vectors to learn from environmental feedback and to select action plans grounded in optimal features. Furthermore, the reference point sampling process utilizes a distributed model to generate novel reference vectors. K. E. Wu et al. [47] introduced a PF tracking approach grounded in B-norm principles. This method involves generating reference points more uniformly across an m-dimensional B-norm surface, which adapts dynamically in response to the true PF being tracked. S. Liu et al. [48] introduced a self-guided reference vector approach, utilizing an enhanced k-means clustering technique to derive reference vectors from the population. Additionally, an angle-based density assessment strategy is employed to dynamically select an adaptive reference vector for guiding the search process. Q. Liu et al. [49] introduced a synchronized adjustment approach for both the reference vector and the aggregation function. Utilizing a local angle threshold, the reference vector is adaptively modified to more closely align with the solution distribution. Furthermore, the weight assigned to the aggregation function is dynamically adjusted based on both the local angle threshold and the age of the reference vector. This coordinated strategy ensures a harmonious balance between diversity and convergence. In the reference point-based approach, the configuration of reference points plays a crucial role in determining the algorithm’s performance. Therefore, the challenge lies in establishing more effective reference points that can efficiently guide the population toward convergence while maintaining diversity, which is essential for the success of this type of algorithm.

2.4. Grid-Based Many-Objective Optimization Algorithms

Knowles and Corne [50,51] introduced the PAES procedure, which updates the optimal non-dominated solution set by utilizing a grid crowding mechanism. This mechanism evaluates the crowding degree of each solution based on the number of solutions occupying the same grid location. Consequently, the solution with the highest crowding degree in the archive set is replaced to ensure the generation of an optimal solution suitable for multi-objective problems. Corne et al. [52] introduced the PESA algorithm, which incorporates a novel crowding mechanism. This algorithm divides the normalized space into super-boxes based on an implicitly constructed super-grid. The objective of this division is to combine the convergence capabilities of evolutionary algorithms with the maintenance of population diversity. Laumanns et al. [4] introduced a grid-based method known as $[eqn]$ -dominance, representing a novel dominance technique. Deb et al. [53] also created a new method of domination, referred to as $[eqn]$ domination. This method divides the objective space into multiple hyperboxes, with each hyperbox containing no more than one individual based on the size of $[eqn]$ . Yen and Lu [54] introduced a novel dynamic multi-objective optimization algorithm. The core concept involves assigning each cell in the grid a rank and a density value based on the grid position of the solution and its Pareto dominance relationship. Knowles and Corne [55] proposed new grid-based archiving schemes. As the points in the objective space are generated and archived, the position and size of the grid in the grid space will be adaptively adjusted so that the grid can surround these points. If the archive set reaches the capacity limit, the grid-based method can well select the points removed from the archive. Rachmawati et al. [56] developed a dynamic method for adjusting the grid, which identifies and adjusts the grid size according to the average occupancy and neighbor occupancy. Y. Zhou et al. [57] developed the ‘box’ function, where the objective space is divided into multiple grids based on the ‘box’ function and a vector is returned by the ‘box’ function to represent a grid vertex. The ‘update’ and ‘box’ functions that are adopted ensure that each grid contains at most one solution. A. G. Hern et al. [58] developed an improved $[eqn]$ -dominance algorithm, which overcomes the problem that some non-dominated solutions are lost from the hyper-grid used in the archive due to the way of selecting solutions in each box in the original method. Karahan and Koksalan [59] introduced a multi-objective optimization algorithm centered on the concept of territory. The core idea involves dividing the objective space into multiple hyperboxes, all of which are of equal size. To prevent overcrowding, each hyperbox is designed to contain at most one individual, and a territory is defined around each individual to maintain high diversity. This approach bears some resemblance to the method proposed in [53]. X. Cai et al. [60] introduced an inverted generational distance metric based on a grid to assess both the convergence and diversity of the population. X. He et al. [61] introduced a method for balancing convergence and diversity through a combination of decomposition-based and Pareto dominance selection based on grids. M. He et al. [62] introduce an evolutionary algorithm leveraging non-uniform grids. Within this system, a dynamic non-uniform meshing approach and an offset-driven density assessment technique are employed to differentiate non-dominated solutions within grid coordinates. V. Palakonda et al. [63] introduced a hybrid approach that partitions the objective space through clustering-based mating selection, utilizing Euclidean distance to identify potential offspring solutions. Subsequently, combined with grid-based one-to-one environment selection, grid coordinate point distance is employed to select the most promising solution. While the grid-based method offers a uniform partition, it tends to result in wasted computing resources, and the algorithm’s performance is significantly affected by the size of the grid.

3. The Proposed Algorithm

3.1. The Main Idea of the Proposed Algorithm

In this paper, we introduce an evolutionary algorithm, specifically designed for solving many-objective optimization problems, called the Reference Point and Grid Method-based Evolutionary Algorithm (RGEA). RGEA combines the benefits of a diversity evaluation method based on reference points, which aids in maintaining population diversity and finding high-quality solutions on the regular Pareto front, with the advantages of introducing grid dominance and grid difference to accelerate the convergence on the irregular Pareto front when evaluating individuals using a grid-based strategy. The ultimate aim is to enhance the algorithm’s capacity to address many-objective optimization problems. RGEA primarily consists of the generation of reference points, normalization, entropy calculation, and grid division. Since entropy serves as a measure to quantify the diversity of solutions in the population for many-objective optimization problems, we incorporate entropy into the algorithm to determine whether to adopt the reference point-based approach or the grid-based approach. In order to calculate the entropy value at each fixed iteration, we introduce a parameter $[eqn]$ to control the interval for calculating the entropy value. By comparing the current entropy value with the maximum entropy value, the search approach based on the reference point or the grid can be determined, thereby ensuring both great convergence and diversity. The structure of the RGEA is outlined below.

The primary concept of Algorithm 1 involves introducing a parameter to regulate the number of iterations, which is used to determine the interval for calculating the entropy value. By comparing the current entropy with the maximum entropy, either the reference point-based approach or the grid-based method is adopted. The union operation is employed to combine the parent population with the offspring population, thereby generating a new population, as shown in Step 3. Then, the new population is non-dominated sorted to select the next population, as shown in Step 4. The population $[eqn]$ consists of $[eqn]$ , where l represents the last layer of the selected non-dominated front. Subsequently, the adaptive normalization process is performed. If the kind is equal to 1, then K individuals are selected from the last non-dominated layer by the niche retention technique, as shown in Steps 14–18. Otherwise, K individuals are chosen from the final non-dominated layer by means of the grid-based method, as shown in Steps 19–22. A parameter $[eqn]$ is introduced for the purpose of controlling the number of iterations, which, in turn, determines the interval for calculating the entropy value. The current entropy and the maximum entropy are compared to determine whether the reference point-based approach or the grid-based approach is adopted. When the current entropy reaches the maximum entropy, the grid-based method is used, as shown in Steps 24–27. The key technologies of RGEA are described in the following sections. Algorithm 1: Framework of RGEAInput:    H structured reference ponits $[eqn]$ or supplied aspiration ponits $[eqn]$     P: population    N: population size    maxGen: maximum number of iterations    N uniformly distributed weight vectors: $[eqn]$     div: the size of grid division     $[eqn]$ : the interval for calculating the entropy valueInitialization:     t = 1     $[eqn]$ = Initialize(P)     Compute maximum entropy: MaxEntropy;Optimization:    while t < maxGen do    1: $[eqn]$ , $[eqn]$ ,    2: $[eqn]$ = Genetic Recombination and Mutation( $[eqn]$ )    3: $[eqn]$     4: $[eqn]$ = Non-dominated Sorting $[eqn]$     5: while $[eqn]$ do    6:    $[eqn]$     7:    $[eqn]$     8: Final Front to Incorporate: $[eqn]$     9: if $[eqn]$ then  10:    $[eqn]$   11: else  12:   $[eqn]$   13:  Points Selected From $[eqn]$   14: if kind = 1  15:   Normalize Objectives and Establish Reference Set $[eqn]$ : $[eqn]$   16:   Associate each member of with a reference point: $[eqn]$   17:   Compute niche count of reference point $[eqn]$   18:   Choose members one at a time from $[eqn]$ to construct          $[eqn]$ : $[eqn]$   19: else  20:    $[eqn]$   21:    $[eqn]$   22:    $[eqn]$   23: end if  24: if $[eqn]$   25:   Calculate CurrentEntropy: $[eqn]$   26:   if MaxEntropy - CurrentEntropy <= 0, then  27:     kind = 2  28: end if  29: end if  30: end if  31: $[eqn]$   32: endOutput:   P

3.2. Generation of Reference Points

By selecting the reference point sets in the objective space, the algorithm benefits from improved stratification and more effective solution selection through the choice of a reference point. The role of the reference point is akin to stratified sampling in statistics, which covers the whole objective space by uniformly distributed reference points, thereby guaranteeing the variety and even dissemination of solutions. When selecting a reference point, three essential criteria must generally be met: 1. Reference points must be evenly distributed within the objective space to ensure comprehensive coverage of the entire space. 2. The number of reference points should be moderate. 3. Having too many reference points can lead to reduced search efficiency. In this paper, the reference point set is generated using the boundary intersection method of DaS and Dennis to construct a fully intelligent approach [22]. This method can generate a uniformly distributed reference point set in the objective space, with a predefined integer representing the division on each coordinate axis. In this approach, reference points are obtained by sampling from a unit, and the quantity of these reference points (denoted as H) can be determined using the following calculation method:

$$[eqn]$$

It can be inferred from Equation (4) that the quantity of reference points is affected by both the dimension M and the integer p associated with the objective. When dealing with a problem involving three objectives and assigning p the value of 4, the total count of reference points becomes H = 15. For a clear explanation, Figure 1a depicts the arrangement of reference points for a single layer on a three-objective space. When p equals M, the number of reference points is considered adequate. However, in the context of a many-objective space, even with p set to M, there is a steep increase in the number of reference points, significantly exacerbating the computational time complexity. For instance, when M is set to 6, then p is equal to 6, which will necessitate the use of 462 reference points. Moreover, if only p is reduced, then this will lead to a sparse distribution of reference points and a loss of diversity among them. To address this issue, the literature [2] has introduced a method of distributing reference points in two layers, utilizing a relatively small p value for the inner points. Assuming that $[eqn]$ and $[eqn]$ represent the outer partition and the inner partition, respectively, then the overall count of reference points (denoted as H) is calculated as follows:

$$[eqn]$$

Figure 1b illustrates the arrangement of reference points for two layers in three objective spaces, with the outer layer delineated by $[eqn]$ = 2 and the inner layer delineated by $[eqn]$ = 3.

3.3. Grid-Based Mechanism

The concept of a grid-based methodology involves dividing the objective space into numerous grids and, subsequently, utilizing these grids to pinpoint the location of an individual in the objective space. In this paper, the adaptive grid construction method [51] was employed to dynamically adjust the size and position of the grids based on the evolving population. Figure 2 shows the grid settings in the k-th objective.

As depicted in Figure 2, $[eqn]$ and $[eqn]$ denote the lowest and highest values of the k-th objective within population P, respectively. According to Formulas (6) and (7), the lower and upper boundaries for the k-th objective are established.

$$[eqn]$$ $$[eqn]$$

where $[eqn]$ signifies the number of divisions in the objective space along each dimension. The lower and upper limits of the mesh in the k-th objective are denoted by $[eqn]$ and $[eqn]$ , respectively. For instance, in Figure 2, $[eqn]$ is equal to 5. Consequently, the objective space is partitioned into $[eqn]$ hypercubes. Therefore, the width $[eqn]$ of the hypercube in the first objective can be calculated using Formula (8).

$$[eqn]$$

Therefore, the individual’s grid position in the k-th objective can be ascertained based on $[eqn]$ and the corresponding width $[eqn]$ . The specific calculation formula is shown in (9):

$$[eqn]$$

In Formula (9), $[eqn]$ denotes the downward integer function, $[eqn]$ stands for the grid coordinates of individual x in the k-th objective, while $[eqn]$ signifies the actual objective value of individual x in the k-th objective. The width of the hypercube in the k-th objective is denoted by $[eqn]$ . As illustrated in Figure 2, the grid coordinates of the individuals from left to right in the k-th objective are 0, 1, 2, 3, and 4, respectively. Taking the bi-objective space in Figure 3 as an example, the grid coordinates of individuals A, B, and C are (1, 0), (1, 2), and (2, 3), respectively.

After determining the grid coordinates of each individual, the grid difference $[eqn]$ between the individuals can be determined. The exact calculation formula is shown in (10):

$$[eqn]$$

In this context, $[eqn]$ and $[eqn]$ denote the grid coordinates of individuals x and y in the k-th objective, respectively, where M represents the dimensionality of the objective space. It is evident that the grid difference is greatly influenced by the partition count, $[eqn]$ , and the range of the grid difference extends from 0 to $[eqn]$ . An increase in the number of partitions, $[eqn]$ , leads to a decrease in the size of each grid cell, which, in turn, results in a larger grid difference.

To facilitate the population’s evolution toward the Pareto front and ensure an even distribution along the PF, the fitness assessment of individuals must encompass two key indicators: convergence and diversity. In this paper, a grid-based approach, utilizing three techniques to assign fitness values to individuals, was employed: grid sorting $[eqn]$ , grid crowding distance $[eqn]$ , and grid coordinate point distance $[eqn]$ .

$[eqn]$ reflects the population’s convergence by ranking individuals based on their grid coordinates across each objective. Specifically, for each individual, $[eqn]$ is calculated as the aggregate of its grid coordinates across all objectives. The exact calculation formula is shown in (11):

$$[eqn]$$

In this context, $[eqn]$ signifies the grid coordinates of individual x for the k-th objective, and M denotes the total number of objectives. Estimating the density of solutions plays a crucial role in allocating fitness, as a well-distributed set of solutions can direct the search toward the entire Pareto front. Consequently, a density estimator for individual x, termed as $[eqn]$ , is introduced. The precise Formula (12) for $[eqn]$ is outlined as follows:

$$[eqn]$$

In this context, $[eqn]$ signifies the grid difference between individual x and individual y, while $[eqn]$ denotes the set of neighbors for individual x. An individual y is considered a neighbor of individual x if $[eqn]$ is less than M. The farther the neighbor position is, the smaller the contribution to it is.

For convergence and diversity metrics, even if $[eqn]$ and $[eqn]$ provide a good reference for the population, there will be cases where individuals cannot be distinguished because they are calculated according to the individual’s grid coordinates and they have an integer value, which indicates that the individual’s $[eqn]$ and $[eqn]$ values may be the same. Here, inspired by the $[eqn]$ -MOEA [49] algorithm, this paper calculated the normalized Euclidean distance between an individual and its corresponding Utopian point within the hypercube, which represents the optimal corner point of that individual’s hypercube and is named $[eqn]$ . The precise Formula (13) for $[eqn]$ is outlined as follows:

$$[eqn]$$

In this context, $[eqn]$ represents the grid coordinates of individual x for the k-th objective, $[eqn]$ stands for the objective value of individual x on the k-th objective, $[eqn]$ denotes the lower boundary of the grid in the k-th objective, and $[eqn]$ signifies the width of the hypercube in the k-th objective. From Formula (13), it is evident that a lower $[eqn]$ is more favorable.

Based on the analysis of the above three indicators $[eqn]$ , $[eqn]$ , and $[eqn]$ , which are used to evaluate the status of the population, it can be seen that they can gradually effectively distinguish individuals. The designated diagram is illustrated in Figure 4.

The fitness assignment based on a grid is mainly conducted to search the optimal solution in the considered frontier. The specific method is to pass the above three criteria. Among all the criteria, the lower value is more desirable. The specific working principle refers to Algorithm 2: Algorithm 2:  $[eqn]$ Input:     $[eqn]$ : optimal solution in P     $[eqn]$ : the i-th solution POptimization:    1: $[eqn]$     2: for $[eqn]$ to $[eqn]$ do    3:   if $[eqn]$ then    4:     $[eqn]$     5:   else if $[eqn]$ then    6:     if $[eqn]$ then    7:       $[eqn]$     8:     else if $[eqn]$ then    9:       if $[eqn]$ then    10:          $[eqn]$     11:       end if    12:     end if    13:   end if    14: end forOutput:          $[eqn]$

3.4. Entropy Calculation

Entropy, a fundamental concept introduced by Claude Shannon in 1948 in the framework of information theory, serves as a metric to assess the degree of uncertainty or lack of clarity in information. Specifically, in the realm of information theory, the entropy $[eqn]$ of a discrete random variable X is defined as follows:

$$[eqn]$$

where $[eqn]$ represents the probability that the random variable X takes the value $[eqn]$ , and N denotes the total number of possible values that X can take. Considering the scenario presented in this paper, N solutions are allocated to K reference vectors, such that each reference vector is associated with $[eqn]$ solutions. Furthermore, the probability of a solution being linked to the i-th reference vector is denoted as $[eqn]$ .

$$[eqn]$$

where, if $[eqn]$ , then $[eqn]$ . When the probability distribution is uniformly distributed, that is $[eqn]$ for all i, then the entropy reaches its maximum value, $[eqn]$ . A higher entropy value indicates a greater degree of information uncertainty and a larger amount of information. In the realm of optimization problems, entropy holds significant applied value. By incorporating the concept of entropy, we can more accurately describe the complexity and uncertainty of the problem, thereby guiding the design and implementation of optimization algorithms. Specifically, in multi-objective optimization problems, entropy serves as a useful metric for assessing the diversity and uniformity of solutions. These problems typically involve multiple objective functions, which may exhibit conflicts and trade-offs. By calculating the entropy of the solutions, we can effectively gauge their diversity and uniformity, thus informing the selection and refinement of the algorithm. The framework for calculating entropy is outlined below.

Algorithm 3 focuses on two main processes: the association operation and entropy calculation. The association operation aims to link each solution in the population with a specific reference point. This is achieved by evaluating the vertical distance between the objective value of each solution and a reference line and then grouping solutions associated with the same reference line into clusters. The entropy calculation, on the other hand, serves to determine the probability distribution of the population, facilitate the selection of an appropriate decomposition algorithm, and enhance population diversity. The association operation involves three key steps, as shown in Steps 1–3: 1. Calculating the distance of each solution to each reference vector. 2. Associating each solution with its nearest reference point. 3. Counting the number of solutions associated with each reference point. Regarding entropy calculation, the process begins with initializing the entropy value to 0, as shown in Step 4. Subsequently, the entropy is computed based on the probability distribution of the solutions, spanning from Step 5 to Step 12. Algorithm 3: Calculate the EntropyInput:     N: The population size     N uniformly distributed weight vectors: $[eqn]$      $[eqn]$ : the count of solutions associated with each reference pointAssociation:    1: Measure the distance between each solution and each reference vector: Cosine    2: Match each solution with its closest reference point: pi    3: Compute the count of solutions associated with each reference point: rhoCalculate the Entropy:    4: entropy = 0    5: i = 1    6: while $[eqn]$ do    7: if $[eqn]$ , then $[eqn]$ , entropy = entropy + $[eqn]$     8: end    9: if the loop ends    10:  entropy = −entropy    11: end if    12: i = i + 1    13: endOutput:     entropy

3.5. Analysis of Computational Complexity

The computational complexity of RGEA is evaluated over a single iteration. Algorithm 1 primarily encompasses three components: non-dominated sorting, environment selection based on either reference points or the grid, and entropy calculation. The non-dominated sorting step has a complexity of $[eqn]$ , where m represents the number of objectives and N denotes the population size. Both the reference point-based and grid-based environment selection steps have a complexity of $[eqn]$ . Additionally, the entropy calculation requires $[eqn]$ computations. On the whole, the overall complexity of RGEA is $[eqn]$ .

4. Experiment Design

4.1. Experiment Settings

The specific experimental configurations are outlined below.

(1)Characteristics of the test problems: For the test problems involving 3, 5, 8, and 10 objectives, the population sizes are set to 91, 210, 156, and 275, respectively. Each algorithm is independently run 20 times on each benchmark. The algorithms considered all have an equal number of evaluations for the same test problem, and the maximum number of fitness evaluations is detailed in Table 1. Additionally, the stopping criterion for these runs is not specified here, but it is presumably based on reaching the maximum number of evaluations listed in Table 1.(2)Configuration of genetic operators: In this paper, our primary focus is on the simulated binary crossover operator and the polynomial mutation operator. We set the crossover probability to 1.0 and the mutation probability to $[eqn]$ , which represents the total number of decision variables. Additionally, the distribution index for both the crossover and mutation operators is set at 20.(3)Parameters in MOEA/D and GrEA: For MOEA/D, the neighborhood size is determined by rounding $[eqn]$ of the population size to the nearest integer for the given benchmark problem. In the case of GrEA, the grid division size follows the guidelines outlined in reference [5].(4)Computation of performance indicators: In this paper, the performance of each algorithm was assessed using the inverted generational distance (IGD) indicator and the hypervolume (HV) indicator, of which the specific calculation reference is in Section 4.4. The average value of these two index values in 20 runs was calculated on each test problem. Another important step in evolutionary computation is to normalize the objective space to a unified scale. The primary approach involves normalizing the objective space to the interval $[eqn]$ based on the minimum and maximum values of each objective, as derived from the true Pareto front of each benchmark problem. Consequently, the normalized ideal point shifts to a zero vector, while the nadir point transforms into (1, 1, …, 1).

4.2. Benchmark

In this paper, two popular continuous extensible multi-objective test problem sets DTLZ [64] and WFG [65] are used. To evaluate the efficacy of RGEA and other comparative algorithms in handling both regular and irregular challenges, this paper utilizes DTLZ1-7 and WFG1-9 test problems, which involve objectives ranging from 3 to 10. Specifically, DTLZ1 satisfies the condition A = E for its Pareto front (PF), while the remaining DTLZ problems adhere to the GFD criterion. For the DTLZ problems, the number of decision variables (n) is determined by the formula n = m + r − 1, where r is set to 5 for DTLZ1 and 10 for the other DTLZ2-6 problems, while r is set to 20 for the DTLZ7 problem. As for the WFG test problems, based on reference [2], the number of decision variables is fixed at n = 24. Additionally, the location-dependent variable k is set to m − 1, and the distance-dependent variable l is set to n − k.

4.3. Algorithms in Comparison

To evaluate the effectiveness of RGEA, this paper considered ten advanced EMO algorithms as comparison algorithms. The implementation of all algorithms was carried out within the PlatEMO framework [66]. The comparison algorithm is introduced as follows.

NSGAIII [2]: By establishing a uniformly distributed reference point beforehand to direct the algorithm’s search, the resulting Pareto front can be made to be as evenly spread as possible while preserving the non-dominated individuals situated close to this reference point.

GrEA [5]: The algorithm utilizes grid division to partition the solution space into multiple grid units, effectively narrowing the search scope and bolstering the search efficiency. These grid units possess inherent attributes that encapsulate both convergence and diversity information. Each individual within the grid occupies a specific position and, by comparing the grid position of an individual with those of others, one can assess its convergence. Furthermore, by counting the number of solutions sharing the same or similar grid positions, the diversity of an individual can be evaluated.

MOEA/D [23]: The algorithm employs the divide-and-conquer approach to transform a multi-objective problem (MOP) into several scalar quantum problems, with each sub-problem formulated by incorporating a set of uniformly distributed weight vectors. The core process involves updating the solutions in proximity to each sub-problem through the use of an aggregation function. This type of algorithm has demonstrated its efficacy in both multi-objective optimization [67] and many-objective optimization scenarios [68].

MOEA/D-DE [67]: This algorithm introduces a novel mechanism, i.e., it modifies the traditional concepts of SBX [69] and the polynomial mutation operator [69]; instead, it proposes the use of the differential evolution (DE) [70] operator for generating offspring. To ensure diversity is maintained, two supplementary measures are implemented.

I-DBEA [24]: This algorithm employs systematic sampling to produce uniformly distributed reference points and connects each solution to a reference point through a straightforward preemptive distance comparison method. To strike a balance between convergence and diversity, two separate distance metrics are utilized.

MSOPSII [71]: This algorithm represents an enhanced version of the MSOPS algorithm. It eliminates the need for initial prior designer intervention by utilizing automatic objective vector generation. Additionally, it simplifies analysis and performs more intricate constraint processing through the adoption of a novel fitness assignment approach.

IBEA [12]: This algorithm employs binary tournaments for mating selection. During the environmental selection process, the objective is to eliminate the weakest individual from the population and iteratively update the fitness values of the remaining solutions.

MOEADAWA [72]: This algorithm builds upon the TCH decomposition method utilized in MOEA/D, introducing a novel concept, i.e., a weight vector initialization method alongside the adaptive allocation of sub-problem weight vectors. This approach aims to achieve greater uniformity within the population.

ANSGAIII [73]: This algorithm incorporates adaptive reference points and a non-dominated sorting technique. The core concept revolves around utilizing adaptive reference points to maintain population diversity and mitigate premature convergence to local optimal solutions.

RVEAa [28]: This algorithm is an enhanced version of RVEA that employs a novel adaptive strategy to dynamically adjust the distribution of reference vectors in accordance with the scale of the objective function.

DEAGNG [74]: This algorithm employs a growing neural gas network to acquire knowledge of the topological structure inherent in the PF. Subsequent adjustments are made to both the reference vectors and the scalarizing functions guided by the topological neighbor relationships that have been elucidated through the learning process.

AdaW [75]: This algorithm initially preserves an archive set containing well-distributed, non-dominated solutions. Subsequently, it constructs reference vectors in unexplored yet promising regions based on the solution distribution within the archive set, while eliminating those located in overcrowded or less favorable areas.

4.4. Selection of Evaluation Metrics

To evaluate and compare the performance of RGEA against the comparison algorithms, this experiment selected the widely used comprehensive performance metrics that simultaneously evaluate convergence and diversity: inverted generational distance (IGD) [76] and hypervolume (HV) [77].

The IGD indicator serves as a metric to quantify the distance between the computed Pareto-approximate solution set and a predefined reference set. Specifically, the reference set comprises points that are uniformly distributed along the Pareto front (PF), whereas the solution set consists of points approximating the PF. Subsequently, the IGD value for the solution set is characterized as follows:

$$[eqn]$$

In this context, $[eqn]$ denotes the Euclidean distance in the objective space between solution v and solution set A. Provided that $[eqn]$ is sufficiently large to adequately represent the Pareto front, $[eqn]$ can serve as a comprehensive metric for evaluating both the convergence and diversity of the solution set. This is performed because, to achieve a lower $[eqn]$ value, solution set A must be sufficiently close to the Pareto front in the objective space, thereby ensuring that every portion of $[eqn]$ has a corresponding solution in A for adequate representation.

Hypervolume (HV): Consider A as the approximate solution set representing the Pareto front, and let $[eqn]$ be a reference point in the objective space. This reference point r is dominated by all the objective vectors within the solution set A. Subsequently, the hypervolume (HV) associated with the reference point r quantifies the volume of the objective space that is dominated by the solution set A and is bounded by the reference point r. The precise Formula (17) for HV is outlined as follows:

$$[eqn]$$

In this context, $[eqn]$ represents the Lebesgue measure, and it is utilized for measuring the volume. Furthermore, $[eqn]$ denotes the hypervolume formed by the reference point $[eqn]$ and the m-th solution in the solution set. The choice of the reference point influences the precision of the hypervolume indicator value to a certain degree. A higher HV indicates a more superior non-dominated solution set.

4.5. Nonparametric Statistics Analysis

In this paper, the Wilcoxon signed rank test [78,79] was employed with a significance level set at $[eqn]$ to demonstrate the statistical significance between the RGEA and ten other algorithms across 20 independent runs. The table highlights the best average results for IGD and HV, which were calculated by each algorithm on each test problem with a blue background. The experimental results utilized three symbols, “+”, “−”, and “=”, to indicate, respectively, when the comparison algorithms outperformed RGEA, when RGEA performed more excellently, and when there was no significant difference between RGEA and the other comparison algorithms.

5. Experimental Results and Analysis

5.1. Comparison with Six Evolutionary Algorithms Without Adaptive Technology

This section shows the IGD values of RGEA and six evolution algorithms without adaptive technology, such as MOEA/D, MOEA/D-DE, NSGAIII, GrEA, MSOPSII, and IBEA, on the DTLZ1-7 and WFG1-9 test problems. The experimental results are shown in Table 2. RGEA performs better than the other six algorithms on all test problems. In the analysis of the IGD metric, RGEA performed best on the DTLZ1 problem with 3 objectives, the DTLZ3 problem with 3 objectives, the DTLZ5 problem with 3 objectives, the WFG2 problem with 3 objectives, the WFG4 problem with 3 and 5 objectives, the WFG6 problem with 3 and 5 objectives, the WFG7 problem with 5 objectives, the WFG8 problem with 3 objectives, and the WFG9 problem with 3 objectives.

When RGEA was compared with MOEA/D, for the regular DTLZ1-4 problems, RGEA performed better on the DTLZ4 problem with 5, 8, and 10 objectives. For the irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5 and DTLZ6 problems with 3 objectives, and it also performed better on the DTLZ7 problem with 3, 5, and 8 objectives. For the WFG1-9 problem with 3-to-10 objectives, RGEA performed better than MOEA/D.

When RGEA is compared MOEA/D-DE, for the regular DTLZ1-4 problems, RGEA performs better on the DTLZ1 and DTLZ3 problems with 3 and 5 objectives, and performs better on the DTLZ4 problem with 5, 8, and 10 objectives. For the DTLZ2 problem with 3-to-10 objectives, RGEA performs better than MOEA/D-DE. For the irregular DTLZ5-7 problems, RGEA performs better on the DTLZ5 problem with three objectives, and performs better on the DTLZ7 problem with three or five objectives. For the WFG1-9 problem with 3-to-10 objectives, RGEA performs better than MOEA/D-DE.

When RGEA was compared with NSGAIII, RGEA performed better on the DTLZ1 problem with 3 objectives and the DTLZ2 and DTLZ3 problems with 8 objectives (with respect to the regular DTLZ1-4 problem). Regarding the irregular DTLZ5-7 problems, RGEA performed better on DTLZ5-7 problems with 3-to-10 objectives. For the irregular WFG1-3 problems, RGEA performed better on the WFG1-3 problems with 3-to-10 objectives. For the regular WFG4-9 problems, RGEA performed better on the WFG8 problem with 8 objectives and the WFG9 problem with 3 objectives.

When RGEA was compared with GrEA, RGEA performed better on the DTLZ1 problem with 3, 8, and 10 objectives; the DTLZ2-3 problems with 3, 5, and 8 objectives; and the DTLZ4 problem with 5 objectives (in terms of the regular DTLZ1-4 problems). Concerning the irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5 problem with 3, 8, and 10 objectives and the DTLZ6 problem with 3 objectives. Regarding the irregular WFG1-3 problems, RGEA performed better on the WFG1 problem with 8 objectives, the WFG2 problem with 3 and 5 objectives, and the WFG3 problem with 10 objectives. For regular WFG4-9 problems, RGEA performed better on the WFG4 and WFG9 problems with 3 objectives; the WFG5-7 problems with 3 and 5 objectives; the WFG8 problems with 3, 5, and 10 objectives; and the WFG9 problem with 3 objectives.

When RGEA was compared with MSOPSII, RGEA performed better on the DTLZ1 and DTLZ3 problems with 3 objectives; the DTLZ2 problems with 3, 5, and 8 objectives; and the DTLZ4 problems with 3-to-10 objectives according to the regular DTLZ1-4 problems. As to the irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5 problem with 3 objectives and the DTLZ7 problems with 3 and 5 objectives. As for the irregular WFG1-3 problems, RGEA performed better on the WFG1 problem with 3 objectives, and it also performed better on the WFG2 problems with 3-to-10 objectives. Regarding regular WFG4-9 problems, RGEA performed better on the WFG4-7 problems with 3, 5, and 8 objectives, and it also performed better on the WFG8-9 problems with 3-to-10 objectives.

When RGEA was compared with IBEA, for the regular DTLZ1-4 problems, RGEA performed better on the DTLZ1-2 problems with 3, 5, and 8 objectives, the DTLZ3 problem with 3 objectives, and the DTLZ4 problem with 5 objectives. For irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5-6 problems with 3 objectives. Regarding irregular WFG1-3 problems, RGEA performed better on WFG2 problems with 3 and 5 objectives. RGEA performed better on WFG4-9 problems with 3, 5, and 8 objectives in terms of the regular WFG4-9 problems. Therefore, the effectiveness of the proposed algorithm RGEA on regular and irregular problems was verified.

In order to show the non-dominated solutions obtained by RGEA and the other six algorithms without adaptive technology on the regular and irregular problems, we selected the DTLZ1, DTLZ5, WFG2, and WFG4 problems with three objectives as representatives to display (corresponding to Figure 5, Figure 6, Figure 7 and Figure 8), as well as the DTLZ1, DTLZ5, WFG2, and WFG8 problems with 10 objectives as representatives to visualize (corresponding to Figure 9, Figure 10, Figure 11 and Figure 12, respectively). As shown in Figure 5, for the regular three-objective DTLZ1 problem, the non-dominated solution set obtained by RGEA was the best, which is similar to the visualization results of MOEA/D, and it was significantly better than NSGA-III, GrEA, MOEA/D-DE, IBEA, and MSOPSII. As shown in Figure 6, for the irregular DTLZ5 problem with three objectives, RGEA obtained the best non-dominated solution set, which had obvious advantages and was similar to the visualization results of NSGA-III. As shown in Figure 7, for the irregular WFG2 problem with three objectives, RGEA obtained the best non-dominated solution set, which had obvious advantages and was similar to the visualization results of NSGA-III. As shown in Figure 8, RGEA was also able to obtain the best visualization results for the WFG4 problem with 3 regular objectives, which is similar to the visualization results of NSGA-III. This shows that RGEA can obtain better results on the regular and irregular DTLZ and WFG problems with 3 objectives. As shown in Figure 9, for the regular 10-objective DTLZ1 problem, the visualization results of RGEA were better than GrEA and were equivalent to MOEA/D-DE, NSGA-III, and IBEA. The visualization result of MOEA/D was the best because the MOEA/D used a decomposition-based method to decompose the multi-objective problem into multiple single-objective problems, which can find non-dominated solutions well. As shown in Figure 10, for the regular 10-objective DTLZ5 problem, MOEA/D obtained the best distribution of non-dominated solutions, MSOPSII and IBEA had similar effects, and RGEA was better than GrEA (similar to NSGAIII). As shown in Figure 11, for the irregular WFG2 problem with 10 objectives, IBEA obtained the best distribution of non-dominated solutions. RGEA was better than MOEA/D, MOEA/D-DE, and MSOPSII, and it was similar to NSGA-III. As shown in Figure 12, RGEA had obvious advantages over MOEA/D, MOEA/D-DE, GrEA, and MSOPSII for the WFG8 problem with 10 objectives, and it was comparable to IBEA, IDBEA, and NSGA-III. It can be seen that RGEA can obtain better non-dominated solution sets on both regular and irregular problems.

5.2. Comparison with Six Evolutionary Algorithms with Adaptive Technology

This section shows the HV values of RGEA and the six evolutionary algorithms with adaptive technology, i.e., as IDBEA, ANSGAIII, MOEA/D-AWA, RVEAa, DEAGNG, and AdaW, on the DTLZ1-7 and WFG1-9 test problems. The experimental results are shown in Table 3. The performance of RGEA was significantly better than IDBEA, ANSGAIII, RVEAa, DEAGNG, and AdaW on all test problems, and it was similar to MOEA/D-AWA. In the HV metric analysis, RGEA had the best performance on the DTLZ1 problem with 3 objectives; the DTLZ4 problem with 5 objectives; the WFG4 problem with 5 objectives; the WFG5 problem with 3, 5, and 8 objectives; the WFG6 problem with 5, 8, and 10 objectives; the WFG7-8 problem with 5 and 8 objectives; and the WFG9 problem with 8 objectives.

When RGEA was compared with IDBEA, RGEA performed better on the DTLZ1 problems with 3, 8, and 10 objectives; on the DTLZ2-3 problems with 3 and 5 objectives; and on the DTLZ4 problem with 5 objectives (with respect to the regular DTLZ1-4 problems). Regarding irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5 problems with 3, 8, and 10 objectives, as well as on the DTLZ7 problem with 3 objectives. Regarding irregular WFG1-3 problems, RGEA performed better on the WFG1 problem with 3 objectives, as well as performed better on the WFG2 problems with 3-to-10 objectives and on the WFG3 problems with 3 and 5 objectives. Concerning regular WFG4-9 problems, RGEA performed better on WFG4-7 problems with 3 and 5 objectives, as well as on the WFG8 problems with 3-to-10 objectives and on the WFG9 problem with 3 objectives.

When RGEA was compared with ANSGAIII, for the regular DTLZ1-4 problems, it performed better on the DTLZ1 and DTLZ3 problems with 3 objectives and on the DTLZ2 problems with 3 and 5 objectives. For the irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5 problem with 8 objectives. For the irregular WFG1-3 problems, RGEA performed better on the WFG1 problem with 5 objectives, as well as on the WFG2 problems with 3 and 5 objectives and on the WFG3 problem with 3 objectives. For the regular WFG4-9 problems, RGEA performed better on the WFG4-7 problems with 3-to-10 objectives, as well as on the WFG8-9 problems with 3, 5, and 8 objectives.

When RGEA was compared with MOEA/D-AWA, RGEA performed better on the DTLZ1 problem with 3 objectives, and it also performs better on the DTLZ2 problem with 8 objectives in terms of regular DTLZ1-4 problems. Concerning irregular DTLZ5-7 problems, RGEA performed better on the DTLZ7 problems with 3 and 5 objectives. Regarding the irregular WFG1-3 problems, RGEA performed better on the WFG2 problem with 3 objectives. Concerning regular WFG4-9 problems, RGEA performed better on the WFG4 problems with 3, 5, and 8 objectives, and it also performed better on the WFG5-9 problems with 3-to-10 objectives.

When RGEA was compared with RVEAa, it performed better on the DTLZ1 and DTLZ3 problems with 3 objectives, the DTLZ2 problems with 3-to-10 objectives and the DTLZ4 problems with 3 and 5 objectives (with respect to the regular DTLZ1-4 problems). For the irregular DTLZ5-7 problems, RGEA performed better on the DTLZ7 problem with 8 objectives. Regarding irregular WFG1-3 problems, RGEA performed better on the WFG1-2 problems with 3-to-10 objectives, and it also performed better on the WFG3 problem with 5 objectives. Concerning regular WFG4-9 problems, RGEA performed well on all problems with 3-to-10 objectives.

When RGEA was compared with DEAGNG, for the regular DTLZ1-4 problems, RGEA performed better on the DTLZ1 problem with 3, 5, and 8 objectives, on the DTLZ2 problem with 3-to-10 objectives, on the DTLZ3 problem with 3 and 5 objectives, and on the DTLZ4 problem with 5 objectives. For the irregular DTLZ5-7 problem, RGEA performed better on the DTLZ5 problem with 8 objectives, and it also performed better on the DTLZ7 problem with 3 objectives. For the irregular WFG1-3 problems, RGEA performed better on the WFG2 problem with 3–10 objectives. For the regular WFG4-9 problem, RGEA performed better on the WFG4 and WFG6-8 problems with 5, 8, and 10 objectives. The performance was better on the WFG5 problem with 3–10 objectives, and the performance was better on the WFG9 problem with 8 and 10 objectives.

When RGEA was compared with AdaW, for the regular DTLZ1-4 problems, RGEA performed better on the DTLZ1 problem with 3, 8, and 10 objectives, and it also performed better on the DTLZ2 and DTLZ4 problems with 5, 8, and 10 objectives. For the irregular DTLZ5-7 problems, RGEA performed better on the DTLZ5 problem with 5, 8, and 10 objectives, and it also performed better on the DTLZ7 problem with 8 and 10 objectives. For irregular WFG1-3 problems, RGEA performed better on WFG2 problems with 10 objectives, and it also performed better on WFG3 problems with 5 objectives. For the regular WFG4-9 problems, RGEA performed better on WFG4-5 and WFG9 problems with 3-to-10 objectives, and it also performed better on WFG6-8 problems with 5, 8, and 10 objectives. Therefore, the effectiveness of the proposed algorithm RGEA on regular and irregular problems was verified.

In order to show the non-dominated solution sets obtained by RGEA and the other six algorithms with adaptive technology on regular and irregular problems, we selected DTLZ1, DTLZ7, WFG2, and WFG6 problems with three objectives as representatives for visualization (corresponding to Figure 13, Figure 14, Figure 15 and Figure 16), as well as the DTLZ4, DTLZ7, WFG2, and WFG6 problems with 10 objectives (corresponding to Figure 17, Figure 18, Figure 19 and Figure 20, respectively). As shown in Figure 13, for the regular three-objective DTLZ1 problem, RGEA obtained the best non-dominated solution set and was significantly better than the other six algorithms. As shown in Figure 14, for the irregular DTLZ7 problem with three objectives, AdaW obtained the best visualization results, and the non-dominated solutions obtained by RGEA were significantly better than IDBEA, MOEA/D-AWA, and DEAGNG, which were similar to those of ANSGAIII. As shown in Figure 15, for the irregular WFG2 problem with 3 objectives, the visualization results obtained by RGEA were significantly better than those of IDBEA, ANSGAIII, MOEA/D-AWA, RVEAa, and DEAGNG, but they were similar to those of AdaW. As shown in Figure 16, for the regular three-objective WFG6 problem, the visualization results obtained by RGEA were significantly better than those of IDBEA, ANSGAIII, MOEA/D-AWA, and RVEAa, but they were similar to those of DEAGNG and AdaW. As shown in Figure 17, for the regular 10-objective DTLZ4 problem, the RGEA visualization results were significantly better than AdaW and ANSGAIII. Although the IDBEA visualization results were similar, this did not apply for RVEAa, MOEA/D-AWA and DEAGNG. As shown in Figure 18, for the irregular DTLZ7 problem with 10 objectives, RGEA had obvious advantages over AdaW and was similar to ANSGAIII, but it was not as good as IDBEA, MOEA/D-AWA, RVEAa, and DEAGNG. As shown in Figure 19, for the irregular WFG2 problem with 10 objectives, the visualization results obtained by RGEA were better than IDBEA, RVEAa, DEAGNG, and AdaW, but they were similar to ANSGAIII and slightly inferior to MOEA/D-AWA. As shown in Figure 20, RGEA was similar to IDBEA for the regular WFG6 problem with 10 objectives but had obvious advantages over the other algorithms. It can be seen that RGEA can obtain better non-dominated solution sets on both regular and irregular problems.

5.3. Analysis of Parameter Sensitivity

In order to study the effect of different values of the parameter $[eqn]$ on the performance of RGEA in this paper, two regular problems and two irregular problems were selected for testing: the DTLZ1, DTLZ5, WFG2, and WFG9 test problems. Each test instance ran 20 times independently. The curves of the average IGD and HV metrics obtained by the proposed algorithm RGEA on each test function with different values of the parameter $[eqn]$ are shown in Figure 21 and Figure 22. From Figure 21 and Figure 22, it can be seen that the different values of parameter $[eqn]$ had little effect on the regular DTLZ1 problem with 3 objectives, the regular WFG9 problems with 3-to-10 objectives, the irregular DTLZ5 problem with 3 objectives, and the irregular WFG2 problems with 3 and 5 objectives. The parameter $[eqn]$ had a significant impact on the regular DTLZ1 problems with 5, 8, and 10 objectives, and it had a great impact on the irregular DTLZ5 problems with 5, 8, and 10 objectives. It had a slight impact on the irregular WFG2 problems with 8 and 10 objectives, which indicates that the parameter $[eqn]$ did not have much impact on the regular and irregular problems with low-dimensional objectives but that it did have a greater impact on the irregular problems with high-dimensional objectives. According to the analysis of the change curve, the parameter $[eqn]$ can achieve relatively good performance at 50, where 50 refers to the entropy value calculated once every 50 iterations. The parameters $[eqn]$ control the interval of the calculated entropy and determine the switching time of the reference point-based method and the grid-based method. When the parameters $[eqn]$ are different, the distribution of offspring generated by the initial reference point-based environment selection method will also be different, which will directly affect the subsequent grid-based environment selection method, thus affecting the entire population. By introducing entropy and parameter $[eqn]$ to determine the switching time of the environment selection method, the current entropy reached the maximum entropy, that is, after determining the number of iterations affected by the parameter $[eqn]$ , the distribution of the population would be the best at that time, and the grid-based method was then switched to accelerate local convergence.

6. Conclusions

In order to solve the problem where the reference point-based method does not obtain good performance on irregular problems and the grid-based method does not obtain good results on regular problems, this paper proposes the Reference Point and Grid Method-based Evolutionary Algorithm (RGEA) with entropy for many-objective optimization problems. In this paper, the parameter $[eqn]$ was introduced to control the interval for calculating the entropy value. By comparing the current entropy with the maximum entropy, the reference point-based method or the grid-based method was determined. When the current entropy reaches the maximum entropy, the grid-based method is adopted. When the current entropy does not reach the maximum entropy, the reference point-based method is adopted. RGEA was compared with six evolutionary algorithms without adaptive technology, i.e., MOEA/D, MOEA/D-DE, NSGAIII, GrEA, MSOPSII, and IBEA, as well as six advanced evolutionary algorithms with adaptive technology, i.e., IDBEA, ANSGAIII, MOEA/D-AWA, RVEAa, DEAGNG, and AdaW. Widely used test problems, such as the DTLZ1-7 and WFG1-9 problems, were selected, and the two types of problems were divided into regular problems and irregular problems when analyzing the specific performance. As verified, RGEA delivered a significant performance in both regular and irregular problems. The main reason why RGEA can effectively deal with regular and irregular problems is that entropy and the parameter $[eqn]$ were introduced into RGEA, which can control the interval of calculation entropy. When RGEA reaches the maximum entropy, the reference point-based method is no longer called; rather, it is converted into a grid-based method, which promotes diversity and convergence. RGEA adopts the reference point-based method to improve the population diversity in the early stage of the search, and it then evolves to a later stage, where the entropy calculation is controlled by the parameter $[eqn]$ . When the current entropy value reaches the maximum entropy value, the grid-based method is switched to improve the convergence. We also studied the sensitivity of the parameter $[eqn]$ , and we concluded that the parameter $[eqn]$ did not have much influence on the regular and irregular problems with low-dimensional objectives, but we did find that it had a greater impact on the irregular problems with high-dimensional objectives. However, this paper only studied the influence of the sensitivity of the parameter $[eqn]$ on the performance of the RGEA algorithm when the entropy calculation interval was between 20 and 60 generations. In follow-up studies, the diversity and convergence of the population in the evolution process of RGEA could be analyzed to determine the number of specific evolution iterations with superior performance in the evolution process, so as to determine a better interval of entropy calculation. At the same time, such studies could also help determine the switching time used in the reference point-based method and the grid-based method so that the RGEA algorithm can show better performance. Therefore, the study of the parameter $[eqn]$ in the evolution process will be the focus of a future research direction.