A PAC Approach to Application-Specific Algorithm Selection

TL;DR

This paper applies statistical and online learning theories to analyze application-specific algorithm selection, providing a theoretical foundation for understanding when empirical and theoretical approaches perform well.

Contribution

It introduces a framework modeling algorithm selection as a statistical learning problem and extends learning theory concepts to broader algorithmic contexts.

Findings

Identifies conditions for guaranteed performance of empirical and theoretical approaches.

Models algorithm selection as a statistical learning problem.

Provides possibility and impossibility results for no-regret algorithms.

Abstract

The best algorithm for a computational problem generally depends on the "relevant inputs," a concept that depends on the application domain and often defies formal articulation. While there is a large literature on empirical approaches to selecting the best algorithm for a given application domain, there has been surprisingly little theoretical analysis of the problem. This paper adapts concepts from statistical and online learning theory to reason about application-specific algorithm selection. Our models capture several state-of-the-art empirical and theoretical approaches to the problem, ranging from self-improving algorithms to empirical performance models, and our results identify conditions under which these approaches are guaranteed to perform well. We present one framework that models algorithm selection as a statistical learning problem, and our work here shows that dimension notions from statistical learning theory, historically used to measure the complexity of classes of binary- and real-valued functions, are relevant in a much broader algorithmic context. We also study the online version of the algorithm selection problem, and give possibility and impossibility results for the existence of no-regret learning algorithms.