Where's Ben Nevis? A 2D optimisation benchmark with 957,174 local optima based on Great Britain terrain data

TL;DR

This paper introduces a new complex benchmark based on Great Britain's terrain data, featuring over 950,000 local optima, to evaluate and compare optimization algorithms' performance in realistic landscapes.

Contribution

The paper presents a novel terrain-based benchmark with a large number of local optima, a classification method for basins, and a framework for benchmarking optimization algorithms.

Findings

Differential Evolution outperformed other algorithms in finding Ben Nevis.

The benchmark effectively differentiates algorithm performance on complex landscapes.

The framework enables meaningful comparisons even when algorithms fail to find the global optimum.

Abstract

We present a novel optimisation benchmark based on the real landscape of Great Britain (GB). The elevation data from the UK Ordnance Survey Terrain 50 dataset is slightly modified and linearly interpolated to produce a target function that simulates the GB terrain, packaged in a new Python module nevis. We introduce a discrete approach to classifying local optima and their corresponding basins of attraction, identifying 957,174 local optima of the target function. We then develop a benchmarking framework for optimisation methods based on this target function, where we propose a Generalised Expected Running Time performance measure to enable meaningful comparisons even when algorithms do not achieve successful runs (find Ben Nevis). Hyperparameter tuning is managed using the optuna framework, and plots and animations are produced to visualise algorithm performance. Using the proposed framework, we benchmark six optimisation algorithms implemented by common Python modules. Amongst those tested, the Differential Evolution algorithm implemented by scipy is the most effective for navigating the complex GB landscape and finding the summit of Ben Nevis.