Deep Boltzmann Machines in Estimation of Distribution Algorithms for Combinatorial Optimization

TL;DR

This paper explores the use of Deep Boltzmann Machines within Estimation of Distribution Algorithms to improve combinatorial optimization, showing advantages on certain complex functions but with higher computational costs.

Contribution

It introduces a novel integration of Deep Boltzmann Machines into EDAs and evaluates its performance on various combinatorial problems.

Findings

DBM-EDA outperforms BOA on complex additively decomposable functions.

DBM-EDA often finds optimal solutions faster on some runs.

Model building is more computationally intensive than other neural network-based EDAs.

Abstract

Estimation of Distribution Algorithms (EDAs) require flexible probability models that can be efficiently learned and sampled. Deep Boltzmann Machines (DBMs) are generative neural networks with these desired properties. We integrate a DBM into an EDA and evaluate the performance of this system in solving combinatorial optimization problems with a single objective. We compare the results to the Bayesian Optimization Algorithm. The performance of DBM-EDA was superior to BOA for difficult additively decomposable functions, i.e., concatenated deceptive traps of higher order. For most other benchmark problems, DBM-EDA cannot clearly outperform BOA, or other neural network-based EDAs. In particular, it often yields optimal solutions for a subset of the runs (with fewer evaluations than BOA), but is unable to provide reliable convergence to the global optimum competitively. At the same time, the model building process is computationally more expensive than that of other EDAs using probabilistic models from the neural network family, such as DAE-EDA.