Procrastinating with Confidence: Near-Optimal, Anytime, Adaptive Algorithm Configuration

TL;DR

This paper introduces an adaptive algorithm configuration method that maintains near-optimality and anytime guarantees, significantly improving performance in practical settings where many configurations perform poorly.

Contribution

It proposes a new algorithm, Structured Procrastination with Confidence, combining adaptivity with near-optimal, anytime performance guarantees.

Findings

The new method is faster in settings with many poor configurations.

Empirical results show frequent occurrence of such settings in practice.

The anytime property helps find good configurations quickly.

Abstract

Algorithm configuration methods optimize the performance of a parameterized heuristic algorithm on a given distribution of problem instances. Recent work introduced an algorithm configuration procedure ("Structured Procrastination") that provably achieves near optimal performance with high probability and with nearly minimal runtime in the worst case. It also offers an $\textit{anytime}$ property: it keeps tightening its optimality guarantees the longer it is run. Unfortunately, Structured Procrastination is not $\textit{adaptive}$ to characteristics of the parameterized algorithm: it treats every input like the worst case. Follow-up work ("LeapsAndBounds") achieves adaptivity but trades away the anytime property. This paper introduces a new algorithm, "Structured Procrastination with Confidence", that preserves the near-optimality and anytime properties of Structured Procrastination while adding adaptivity. In particular, the new algorithm will perform dramatically faster in settings where many algorithm configurations perform poorly. We show empirically both that such settings arise frequently in practice and that the anytime property is useful for finding good configurations quickly.