Automated Metaheuristic Algorithm Design with Autoregressive Learning

TL;DR

This paper introduces an autoregressive learning-based approach for automated metaheuristic algorithm design, enabling diverse, structure-discovering algorithms that outperform human-designed baselines across multiple benchmarks.

Contribution

It proposes a novel autoregressive generative network to design metaheuristics as sequence generation, allowing for diversity, continual learning, and improved performance over existing methods.

Findings

Outperforms all human baselines on 24 out of 25 problems

Generates diverse algorithms with various structures

Enables continual learning for future problem adaptation

Abstract

Automated design of metaheuristic algorithms offers an attractive avenue to reduce human effort and gain enhanced performance beyond human intuition. Current automated methods design algorithms within a fixed structure and operate from scratch. This poses a clear gap towards fully discovering potentials over the metaheuristic family and fertilizing from prior design experience. To bridge the gap, this paper proposes an autoregressive learning-based designer for automated design of metaheuristic algorithms. Our designer formulates metaheuristic algorithm design as a sequence generation task, and harnesses an autoregressive generative network to handle the task. This offers two advances. First, through autoregressive inference, the designer generates algorithms with diverse lengths and structures, enabling to fully discover potentials over the metaheuristic family. Second, prior design knowledge learned and accumulated in neurons of the designer can be retrieved for designing algorithms for future problems, paving the way to continual design of algorithms for open-ended problem-solving. Extensive experiments on numeral benchmarks and real-world problems reveal that the proposed designer generates algorithms that outperform all human-created baselines on 24 out of 25 test problems. The generated algorithms display various structures and behaviors, reasonably fitting for different problem-solving contexts. Code will be released after paper publication.