An Analysis of Quality Indicators Using Approximated Optimal Distributions in a Three-dimensional Objective Space

TL;DR

This paper investigates the optimal distributions of objective vectors for nine quality indicators in multi-objective optimization, revealing that uniform distributions are often suboptimal and each indicator has its own optimal distribution.

Contribution

It formulates a method to approximate optimal distributions for quality indicators and analyzes their properties across various Pareto fronts, providing new insights into indicator behaviors.

Findings

Uniform distributions are often not optimal for quality indicators.

Each indicator has a unique optimal distribution depending on the Pareto front.

The analysis enhances understanding of indicator properties and their suitability.

Abstract

Although quality indicators play a crucial role in benchmarking evolutionary multi-objective optimization algorithms, their properties are still unclear. One promising approach for understanding quality indicators is the use of the optimal distribution of objective vectors that optimizes each quality indicator. However, it is difficult to obtain the optimal distribution for each quality indicator, especially when its theoretical property is unknown. Thus, optimal distributions for most quality indicators have not been well investigated. To address these issues, first, we propose a problem formulation of finding the optimal distribution for each quality indicator on an arbitrary Pareto front. Then, we approximate the optimal distributions for nine quality indicators using the proposed problem formulation. We analyze the nine quality indicators using their approximated optimal distributions on eight types of Pareto fronts of three-objective problems. Our analysis demonstrates that uniformly-distributed objective vectors over the entire Pareto front are not optimal in many cases. Each quality indicator has its own optimal distribution for each Pareto front. We also examine the consistency among the nine quality indicators.