Quality Indicators for Preference-based Evolutionary Multi-objective Optimization Using a Reference Point: A Review and Analysis

TL;DR

This paper reviews and analyzes various quality indicators used for benchmarking preference-based evolutionary multi-objective optimization with reference points, highlighting their properties, limitations, and impact on algorithm ranking.

Contribution

It provides a systematic review of existing quality indicators, investigates their properties, and discusses how their differences affect benchmarking and decision-making.

Findings

Achievement scalarizing function may not reflect true distance to reference point.

Regions of interest vary with reference point position and Pareto front shape.

Some quality indicators have undesirable properties affecting algorithm ranking.

Abstract

Some quality indicators have been proposed for benchmarking preference-based evolutionary multi-objective optimization algorithms using a reference point. Although a systematic review and analysis of the quality indicators are helpful for both benchmarking and practical decision-making, neither has been conducted. In this context, first, this paper reviews existing regions of interest and quality indicators for preference-based evolutionary multi-objective optimization using the reference point. We point out that each quality indicator was designed for a different region of interest. Then, this paper investigates the properties of the quality indicators. We demonstrate that an achievement scalarizing function value is not always consistent with the distance from a solution to the reference point in the objective space. We observe that the regions of interest can be significantly different depending on the position of the reference point and the shape of the Pareto front. We identify undesirable properties of some quality indicators. We also show that the ranking of preference-based evolutionary multi-objective optimization algorithms depends on the choice of quality indicators.