Searching for a practical evidence of the No Free Lunch theorems

TL;DR

This paper introduces an evolutionary method to generate specific test functions where a particular optimization algorithm outperforms others, providing practical evidence related to the No Free Lunch theorems.

Contribution

It proposes a novel evolutionary approach to evolve test functions that favor a chosen algorithm over others, addressing a key challenge in NFL-related research.

Findings

Successfully evolved functions where Random Search outperforms other algorithms

Demonstrated effectiveness of the evolutionary approach in generating targeted test functions

Validated the approach through multiple numerical experiments

Abstract

According to the No Free Lunch (NFL) theorems all black-box algorithms perform equally well when compared over the entire set of optimization problems. An important problem related to NFL is finding a test problem for which a given algorithm is better than another given algorithm. Of high interest is finding a function for which Random Search is better than another standard evolutionary algorithm. In this paper, we propose an evolutionary approach for solving this problem: we will evolve test functions for which a given algorithm A is better than another given algorithm B. Two ways for representing the evolved functions are employed: as GP trees and as binary strings. Several numerical experiments involving NFL-style Evolutionary Algorithms for function optimization are performed. The results show the effectiveness of the proposed approach. Several test functions for which Random Search performs better than all other considered algorithms have been evolved.