Stein Variational Black-Box Combinatorial Optimization

TL;DR

This paper introduces a Stein variational approach for black-box combinatorial optimization, promoting exploration of multiple optima and outperforming existing methods on large-scale problems.

Contribution

The work integrates the Stein operator into EDAs to enhance exploration and avoid premature convergence in high-dimensional, multimodal black-box optimization.

Findings

Achieves competitive or superior performance on benchmark problems.

Effectively explores multiple modes in complex landscapes.

Excels particularly on large-scale instances.

Abstract

Combinatorial black-box optimization in high-dimensional settings demands a careful trade-off between exploiting promising regions of the search space and preserving sufficient exploration to identify multiple optima. Although Estimation-of-Distribution Algorithms (EDAs) provide a powerful model-based framework, they often concentrate on a single region of interest, which may result in premature convergence when facing complex or multimodal objective landscapes. In this work, we incorporate the Stein operator to introduce a repulsive mechanism among particles in the parameter space, thereby encouraging the population to disperse and jointly explore several modes of the fitness landscape. Empirical evaluations across diverse benchmark problems show that the proposed method achieves performance competitive with, and in several cases superior to, leading state-of-the-art approaches, particularly on large-scale instances. These findings highlight the potential of Stein variational gradient descent as a promising direction for addressing large, computationally expensive, discrete black-box optimization problems.