Reproducibility in Evolutionary Computation

TL;DR

This paper discusses the importance of reproducibility in Evolutionary Computation, classifies types of reproducibility, identifies obstacles, and proposes guidelines and tools to improve reproducibility practices.

Contribution

It offers a refined classification of reproducibility in EC, analyzes obstacles, and provides practical guidelines and tools to enhance reproducibility in the field.

Findings

Classification of reproducibility types in EC

Identification of cultural and technical obstacles

Proposed guidelines and tools for reproducibility

Abstract

Experimental studies are prevalent in Evolutionary Computation (EC), and concerns about the reproducibility and replicability of such studies have increased in recent times, reflecting similar concerns in other scientific fields. In this article, we discuss, within the context of EC, the different types of reproducibility and suggest a classification that refines the badge system of the Association of Computing Machinery (ACM) adopted by ACM Transactions on Evolutionary Learning and Optimization (https://dlnext.acm.org/journal/telo). We identify cultural and technical obstacles to reproducibility in the EC field. Finally, we provide guidelines and suggest tools that may help to overcome some of these reproducibility obstacles.