Relation Reasoning with LLMs in Expensive Optimization

TL;DR

This paper introduces R2SAEA, a relation-based LLM surrogate for expensive black-box optimization, utilizing reinforcement learning and efficient inference strategies to outperform existing methods.

Contribution

It proposes a novel relation reasoning approach with an anchor-based inference method and RL fine-tuning, achieving state-of-the-art results without frequent retraining.

Findings

R2SAEA improves relation prediction accuracy.

It achieves superior optimization performance on benchmarks.

Quantization enables efficient edge deployment.

Abstract

Expensive optimization problems (EOPs) are black-box tasks with costly objective evaluations and no gradient access, making the evaluation budget the key bottleneck. Surrogate-assisted evolutionary algorithms (SAEAs) reduce evaluations via surrogate predictions, but conventional surrogates often require frequent retraining as populations evolve, incurring overhead. This paper proposes R2SAEA, a reinforcement-trained relation-based large language model (LLM) surrogate assisted evolutionary algorithm. We cast relation-based surrogate modeling as an in-context pairwise reasoning task. To enable efficient inference in evolutionary loops, we develop an anchor-based iterative context construction strategy that reduces prompt complexity from quadratic to linear in population size, and a voting-based aggregation scheme that converts predicted relations into scores for offspring selection. We further build an RL pipeline from evolutionary trajectories and fine-tune Qwen2.5 with GRPO. Experiments on single- and multi-objective benchmarks show improved relation prediction and state-of-the-art optimization performance over strong SAEA baselines and general LLMs. Quantization also enables efficient edge deployment, supporting a zero-shot surrogate paradigm without per-generation retraining. Code and models are available at https://github.com/Septend9/R2SAEA.