When is local search both effective and efficient?

TL;DR

This paper investigates when local search algorithms are both effective and efficient on combinatorial optimization problems, introducing new landscape classifications and analyzing the efficiency of various local search methods.

Contribution

It introduces the concept of conditionally-smooth fitness landscapes and analyzes the efficiency of different local search algorithms within this framework.

Findings

Efficient algorithms include random ascent, simulated annealing, and Kernighan-Lin heuristic.

Steepest ascent and random facet may require super-polynomial steps.

Condtionally-smooth landscapes are recognizable in polynomial time.

Abstract

Combinatorial optimization problems implicitly define fitness landscapes that combine the numeric structure of the 'fitness' function to be maximized with the combinatorial structure of which assignments are 'adjacent'. Local search starts at an assignment in this landscape and successively moves assignments until no further improvement is possible among the adjacent assignments. Classic analyses of local search algorithms have focused more on the question of effectiveness ("did we find a good solution?") and often implicitly assumed that there are no doubts about their efficiency ("did we find it quickly?"). But there are many reasons to doubt the efficiency of local search. Even if we focus on fitness landscapes on the hypercube that are single peaked on every subcube (i.e., semismooth fitness landscapes) where effectiveness is obvious, many local search algorithms are known to be inefficient. Since fitness landscapes are unwieldy exponentially large objects, we focus on their polynomial-sized representations by instances of valued constraint satisfaction problems (VCSP). We define a "direction" for valued constraints such that directed VCSPs generate semismooth fitness landscapes. We call VCSPs oriented if they do not have any pair of variables with arcs in both directions. Since recognizing if a VCSP-instance is directed or oriented is coNP-complete, we generalized oriented VCSPs as conditionally-smooth fitness landscapes that are recognizable in polynomial time for a VCSP-instance. We prove that many popular local search algorithms like random ascent, simulated annealing, history-based rules, jumping rules, and the Kernighan-Lin heuristic are very efficient on conditionally-smooth landscapes. But conditionally-smooth landscapes are still expressive enough so that algorithms like steepest ascent and random facet require a super-polynomial number of steps to find the fitness peak.