Large Language Model-Based Benchmarking Experiment Settings for Evolutionary Multi-Objective Optimization

TL;DR

This paper investigates how large language models design benchmarking experiments for evolutionary multi-objective optimization algorithms, revealing they tend to suggest classical settings like standard test problems and performance metrics.

Contribution

It is the first study to analyze the implicit assumptions made by LLMs when designing EMO benchmarking experiments, highlighting their tendency to favor traditional configurations.

Findings

LLMs often recommend classical benchmark problems such as ZDT, DTLZ, and WFG.

Standard performance metrics like HV and IGD are commonly suggested by LLMs.

Designed experiments by LLMs align with conventional EMO evaluation practices.

Abstract

When we manually design an evolutionary optimization algorithm, we implicitly or explicitly assume a set of target optimization problems. In the case of automated algorithm design, target optimization problems are usually explicitly shown. Recently, the use of large language models (LLMs) for the design of evolutionary multi-objective optimization (EMO) algorithms have been examined in some studies. In those studies, target multi-objective problems are not always explicitly shown. It is well known in the EMO community that the performance evaluation results of EMO algorithms depend on not only test problems but also many other factors such as performance indicators, reference point, termination condition, and population size. Thus, it is likely that the designed EMO algorithms by LLMs depends on those factors. In this paper, we try to examine the implicit assumption about the performance comparison of EMO algorithms in LLMs. For this purpose, we ask LLMs to design a benchmarking experiment of EMO algorithms. Our experiments show that LLMs often suggest classical benchmark settings: Performance examination of NSGA-II, MOEA/D and NSGA-III on ZDT, DTLZ and WFG by HV and IGD under the standard parameter specifications.