Explorative Data Analysis of Time Series based AlgorithmFeatures of CMA-ES Variants

TL;DR

This paper explores the use of time-series features to analyze and predict the behavior and performance of CMA-ES algorithm variants across different optimization problems, revealing insights into their search dynamics.

Contribution

It introduces a method for extracting and selecting time-series features to classify CMA-ES variants and predict their performance, highlighting the impact of time series length on predictive power.

Findings

Features can classify CMA-ES variants and problem groups effectively.

Longer time series improve the predictive accuracy of algorithm performance.

Clustering results vary significantly with different cutoff times, indicating behavioral changes.

Abstract

In this study, we analyze behaviours of the well-known CMA-ES by extracting the time-series features on its dynamic strategy parameters. An extensive experiment was conducted on twelve CMA-ES variants and 24 test problems taken from the BBOB (Black-Box Optimization Bench-marking) testbed, where we used two different cutoff times to stop those variants. We utilized the tsfresh package for extracting the features and performed the feature selection procedure using the Boruta algorithm, resulting in 32 features to distinguish either CMA-ES variants or the problems. After measuring the number of predefined targets reached by those variants, we contrive to predict those measured values on each test problem using the feature. From our analysis, we saw that the features can classify the CMA-ES variants, or the function groups decently, and show a potential for predicting the performance of those variants. We conducted a hierarchical clustering analysis on the test problems and noticed a drastic change in the clustering outcome when comparing the longer cutoff time to the shorter one, indicating a huge change in search behaviour of the algorithm. In general, we found that with longer time series, the predictive power of the time series features increase.