Iterated Local Search with Linkage Learning

TL;DR

This paper introduces an enhanced local search method that constructs weighted variable interaction graphs, capturing interaction strengths, which aids in understanding problem structure and designing better optimization operators.

Contribution

It proposes local search with linkage learning 2, capable of efficiently building weighted interaction graphs that reveal interaction strengths and support new operator design.

Findings

Efficiently constructs weighted variable interaction graphs.

Provides insights into feature interactions in machine learning.

Enables design of new perturbation operators for optimization.

Abstract

In pseudo-Boolean optimization, a variable interaction graph represents variables as vertices, and interactions between pairs of variables as edges. In black-box optimization, the variable interaction graph may be at least partially discovered by using empirical linkage learning techniques. These methods never report false variable interactions, but they are computationally expensive. The recently proposed local search with linkage learning discovers the partial variable interaction graph as a side-effect of iterated local search. However, information about the strength of the interactions is not learned by the algorithm. We propose local search with linkage learning 2, which builds a weighted variable interaction graph that stores information about the strength of the interaction between variables. The weighted variable interaction graph can provide new insights about the optimization problem and behavior of optimizers. Experiments with NK landscapes, knapsack problem, and feature selection show that local search with linkage learning 2 is able to efficiently build weighted variable interaction graphs. In particular, experiments with feature selection show that the weighted variable interaction graphs can be used for visualizing the feature interactions in machine learning. Additionally, new transformation operators that exploit the interactions between variables can be designed. We illustrate this ability by proposing a new perturbation operator for iterated local search.