Partial Reinitialisation for Optimisers

TL;DR

This paper introduces a partial reinitialization technique for heuristic optimizers, which improves solution quality by reinitializing only subsets of variables rather than entire configurations, retaining valuable information from previous runs.

Contribution

The paper presents a novel partial reinitialization method that enhances heuristic optimizer performance by selectively reinitializing variable subsets, outperforming standard full reinitialization approaches.

Findings

Significant improvements in solution quality across various problems

Retains more information from previous runs

Faster convergence to better solutions

Abstract

Heuristic optimisers which search for an optimal configuration of variables relative to an objective function often get stuck in local optima where the algorithm is unable to find further improvement. The standard approach to circumvent this problem involves periodically restarting the algorithm from random initial configurations when no further improvement can be found. We propose a method of partial reinitialization, whereby, in an attempt to find a better solution, only sub-sets of variables are re-initialised rather than the whole configuration. Much of the information gained from previous runs is hence retained. This leads to significant improvements in the quality of the solution found in a given time for a variety of optimisation problems in machine learning.