Ergodic Annealing

TL;DR

This paper introduces the Macau Algorithm, a reinforcement learning-based variation of Simulated Annealing, enabling effective optimization when the cost function is unknown and must be learned by an agent.

Contribution

It replaces the Metropolis engine in Simulated Annealing with reinforcement learning, extending its applicability to unknown cost functions.

Findings

The Macau Algorithm effectively learns cost functions during optimization.

It outperforms traditional methods in unknown-cost scenarios.

Reinforcement learning enhances Simulated Annealing's flexibility.

Abstract

Simulated Annealing is the crowning glory of Markov Chain Monte Carlo Methods for the solution of NP-hard optimization problems in which the cost function is known. Here, by replacing the Metropolis engine of Simulated Annealing with a reinforcement learning variation -- that we call Macau Algorithm -- we show that the Simulated Annealing heuristic can be very effective also when the cost function is unknown and has to be learned by an artificial agent.