A Complementarity Analysis of the COCO Benchmark Problems and Artificially Generated Problems

TL;DR

This paper compares artificially generated benchmark problems with the COCO set using landscape analysis and visualization, aiming to reduce bias and improve the representativeness of benchmark problems for optimization algorithms.

Contribution

It introduces a method to analyze and compare artificial and real benchmark problems using landscape analysis and visualization techniques.

Findings

Artificial problems can be effectively analyzed using landscape features.

Visualization reveals relations and differences between problem sets.

Analysis helps in reducing bias in benchmark problem selection.

Abstract

When designing a benchmark problem set, it is important to create a set of benchmark problems that are a good generalization of the set of all possible problems. One possible way of easing this difficult task is by using artificially generated problems. In this paper, one such single-objective continuous problem generation approach is analyzed and compared with the COCO benchmark problem set, a well know problem set for benchmarking numerical optimization algorithms. Using Exploratory Landscape Analysis and Singular Value Decomposition, we show that such representations allow us to further explore the relations between the problems by applying visualization and correlation analysis techniques, with the goal of decreasing the bias in benchmark problem assessment.