Adapting Zeroth Order Algorithms for Comparison-Based Optimization

TL;DR

This paper introduces a method to adapt existing Zeroth-Order Optimization algorithms for Comparison-Based Optimization, expanding available tools for limited-access objective functions and benchmarking their performance.

Contribution

Proposes a simple conversion technique from ZOO to CBO algorithms, enabling broader application and comparison of optimization methods under limited information conditions.

Findings

Converted ZOO algorithms to CBO with promising results

Benchmarking shows effectiveness of adapted algorithms

Hyperparameter tuning improves performance

Abstract

Comparison-Based Optimization (CBO) is an optimization paradigm that assumes only very limited access to the objective function f(x). Despite the growing relevance of CBO to real-world applications, this field has received little attention as compared to the adjacent field of Zeroth-Order Optimization (ZOO). In this work we propose a relatively simple method for converting ZOO algorithms to CBO algorithms, thus greatly enlarging the pool of known algorithms for CBO. Via PyCUTEst, we benchmarked these algorithms against a suite of unconstrained problems. We then used hyperparameter tuning to determine optimal values of the parameters of certain algorithms, and utilized visualization tools such as heat maps and line graphs for purposes of interpretation. All our code is available at https://github.com/ishaslavin/Comparison_Based_Optimization.