Expensive Optimisation: A Metaheuristics Perspective

TL;DR

This paper explores three surrogate-assisted evolutionary algorithms to reduce evaluation costs in expensive optimization problems, comparing their effectiveness and limitations on benchmark functions.

Contribution

It introduces and compares three surrogate-based methods, including support vector machine models and preference learning, for efficient optimization under high evaluation costs.

Findings

DAFHEA reduces evaluation count with SVM surrogates.

DAFHEA II adapts to uncertain environments.

Surrogate ranking effectively estimates candidate fitness.

Abstract

Stochastic, iterative search methods such as Evolutionary Algorithms (EAs) are proven to be efficient optimizers. However, they require evaluation of the candidate solutions which may be prohibitively expensive in many real world optimization problems. Use of approximate models or surrogates is being explored as a way to reduce the number of such evaluations. In this paper we investigated three such methods. The first method (DAFHEA) partially replaces an expensive function evaluation by its approximate model. The approximation is realized with support vector machine (SVM) regression models. The second method (DAFHEA II) is an enhancement on DAFHEA to accommodate for uncertain environments. The third one uses surrogate ranking with preference learning or ordinal regression. The fitness of the candidates is estimated by modeling their rank. The techniques' performances on some of the benchmark numerical optimization problems have been reported. The comparative benefits and shortcomings of both techniques have been identified.