Quality Measures of Parameter Tuning for Aggregated Multi-Objective Temporal Planning

TL;DR

This paper investigates how hypervolume can be a more effective quality measure than best fitness for parameter tuning in multi-objective temporal planning, demonstrating its advantages through empirical case studies.

Contribution

It introduces the use of hypervolume as a target metric for parameter tuning in multi-objective aggregation, showing its superiority over traditional fitness measures.

Findings

Hypervolume-based tuning outperforms fitness-based tuning in the case study.

ParamILS effectively distinguishes between the two approaches.

Using hypervolume improves overall performance in multi-objective temporal planning.

Abstract

Parameter tuning is recognized today as a crucial ingredient when tackling an optimization problem. Several meta-optimization methods have been proposed to find the best parameter set for a given optimization algorithm and (set of) problem instances. When the objective of the optimization is some scalar quality of the solution given by the target algorithm, this quality is also used as the basis for the quality of parameter sets. But in the case of multi-objective optimization by aggregation, the set of solutions is given by several single-objective runs with different weights on the objectives, and it turns out that the hypervolume of the final population of each single-objective run might be a better indicator of the global performance of the aggregation method than the best fitness in its population. This paper discusses this issue on a case study in multi-objective temporal planning using the evolutionary planner DaE-YAHSP and the meta-optimizer ParamILS. The results clearly show how ParamILS makes a difference between both approaches, and demonstrate that indeed, in this context, using the hypervolume indicator as ParamILS target is the best choice. Other issues pertaining to parameter tuning in the proposed context are also discussed.