Curse of scale-freeness: Intractability of large-scale optimization with multi-start methods

TL;DR

This paper reveals that large-scale multi-start optimization methods face fundamental intractability issues due to scale-free phenomena, making it extremely challenging to improve solutions efficiently as problem size grows.

Contribution

The paper provides a theoretical analysis of the intractability of large-scale multi-start optimization, introducing power-law formulas and the concept of scale-freeness in expected improvement metrics.

Findings

Expected relative gap exhibits scale-freeness as a function of iterations.

Half-life of the expected relative gap is proportional to iterations.

Overcoming the curse requires advanced local search with exponential acceleration.

Abstract

This paper investigates the intractability of large-scale optimization with multi-start methods. For the theoretical performance analysis, we focus on random multi-start (RMS), which is one of the representative multi-start methods, including RMS local search and greedy randomized adaptive search procedure (GRASP). Our primary theoretical contribution is to derive, by using extreme value theory, power-law formulas for the two quantities: (i) the expected improvement rate of the best empirical objective value (EOV); (ii) the expected relative gap between the best EOV and the supremum of the EOVs. Notably, the expected relative gap exhibits scale-freeness as a function of the number of iterations. Consequently, the half-life of the expected relative gap is asymptotically proportional to the number of iterations executed by the RMS method. This result can be interpreted as the curse of scale-freeness -- a Zeno's paradox-like phenomenon -- expressed by the metaphor "Reaching for the goal makes it slip away." Through numerical experiments, we observe that several RMS algorithms applied to traveling salesman problems suffer from the curse of scale-freeness. Furthermore, we show that overcoming this curse requires a powerful local search algorithm with effective restart and diversification strategies that exponentially accelerate solution improvement relative to the RMS method.