COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting

TL;DR

COCO is an open source benchmarking platform designed to systematically compare continuous optimization algorithms, including deterministic and stochastic solvers, in a black-box setting for single and multi-objective problems.

Contribution

The paper introduces COCO, a comprehensive platform with standardized methodology for benchmarking continuous optimizers, facilitating reproducibility and fair comparison across diverse algorithms.

Findings

COCO enables automated benchmarking of various optimization algorithms.

The platform supports both deterministic and stochastic solvers.

It provides guidelines for effective benchmarking practices.

Abstract

We introduce COCO, an open source platform for Comparing Continuous Optimizers in a black-box setting. COCO aims at automatizing the tedious and repetitive task of benchmarking numerical optimization algorithms to the greatest possible extent. The platform and the underlying methodology allow to benchmark in the same framework deterministic and stochastic solvers for both single and multiobjective optimization. We present the rationales behind the (decade-long) development of the platform as a general proposition for guidelines towards better benchmarking. We detail underlying fundamental concepts of COCO such as the definition of a problem as a function instance, the underlying idea of instances, the use of target values, and runtime defined by the number of function calls as the central performance measure. Finally, we give a quick overview of the basic code structure and the currently available test suites.