Benchmarking Stopping Criteria for Evolutionary Multi-objective Optimization

TL;DR

This paper introduces a new benchmarking methodology for stopping criteria in evolutionary multi-objective optimization, including a performance measure, a file-based approach, and an efficient data representation.

Contribution

It proposes a comprehensive benchmarking framework with a performance measure, file-based approach, and data representation for evaluating EMO stopping criteria.

Findings

The performance measure simplifies comparison of stopping criteria.

The file-based approach enhances reproducibility and ease of benchmarking.

The data representation reduces file size issues in benchmarking.

Abstract

Stopping criteria automatically determine when to stop an evolutionary algorithm, so as not to waste function evaluations on a stagnant population. Although stopping criteria play an important role in real-world applications, they have attracted little attention in the evolutionary multi-objective optimization (EMO) community. In fact, new stopping criteria for EMO have been rarely developed in recent years. One reason for the stagnation in developing stopping criteria for EMO is a lack of effective benchmarking methodologies. To address this issue, this paper proposes (i) a performance measure of stopping criteria for EMO and (ii) a file-based benchmarking approach. This paper also proposes (iii) a data representation method that effectively stores population states in text files. (i) The proposed measure represents the performance of stopping criteria as a single scalar value, making comparison easy. (ii) The proposed file-based approach not only simplifies the benchmarking process but also facilitates reproducibility. (iii) The proposed data representation method addresses the issue of file size in (ii). We demonstrate the effectiveness of our three contributions (i)--(iii) by benchmarking five representative stopping criteria for EMO.