Reproducibility and Baseline Reporting for Dynamic Multi-objective Benchmark Problems

TL;DR

This paper introduces a reproducibility framework for dynamic multi-objective optimization problems, enabling consistent evaluation of algorithms across diverse dynamic settings and establishing baseline performance metrics.

Contribution

It extends PlatEMO to support reproducible experiments on DMOPs and proposes a baseline schema to evaluate and compare dynamic and non-dynamic evolutionary algorithms.

Findings

Non-dynamic algorithms can be improved with simple diversity strategies.

Baseline performance of non-dynamic algorithms sets a minimum for dynamic algorithm effectiveness.

Framework facilitates reproducibility and comprehensive evaluation of DMOP algorithms.

Abstract

Dynamic multi-objective optimization problems (DMOPs) are widely accepted to be more challenging than stationary problems due to the time-dependent nature of the objective functions and/or constraints. Evaluation of purpose-built algorithms for DMOPs is often performed on narrow selections of dynamic instances with differing change magnitude and frequency or a limited selection of problems. In this paper, we focus on the reproducibility of simulation experiments for parameters of DMOPs. Our framework is based on an extension of PlatEMO, allowing for the reproduction of results and performance measurements across a range of dynamic settings and problems. A baseline schema for dynamic algorithm evaluation is introduced, which provides a mechanism to interrogate performance and optimization behaviours of well-known evolutionary algorithms that were not designed specifically for DMOPs. Importantly, by determining the maximum capability of non-dynamic multi-objective evolutionary algorithms, we can establish the minimum capability required of purpose-built dynamic algorithms to be useful. The simplest modifications to manage dynamic changes introduce diversity. Allowing non-dynamic algorithms to incorporate mutated/random solutions after change events determines the improvement possible with minor algorithm modifications. Future expansion to include current dynamic algorithms will enable reproduction of their results and verification of their abilities and performance across DMOP benchmark space.