Accelerating ERM for data-driven algorithm design using output-sensitive techniques

TL;DR

This paper introduces output-sensitive algorithms that efficiently enumerate the piecewise structure of dual loss functions in data-driven algorithm design, enabling faster empirical risk minimization for combinatorial problems with multiple parameters.

Contribution

It develops novel output-sensitive geometric algorithms and execution graphs to improve the efficiency of ERM in data-driven algorithm design for complex combinatorial problems.

Findings

Efficient enumeration of loss function pieces scales with actual complexity.

Algorithms applied to pricing, clustering, and sequence alignment.

Significant reduction in computational overhead for ERM tasks.

Abstract

Data-driven algorithm design is a promising, learning-based approach for beyond worst-case analysis of algorithms with tunable parameters. An important open problem is the design of computationally efficient data-driven algorithms for combinatorial algorithm families with multiple parameters. As one fixes the problem instance and varies the parameters, the "dual" loss function typically has a piecewise-decomposable structure, i.e. is well-behaved except at certain sharp transition boundaries. In this work we initiate the study of techniques to develop efficient ERM learning algorithms for data-driven algorithm design by enumerating the pieces of the sum dual loss functions for a collection of problem instances. The running time of our approach scales with the actual number of pieces that appear as opposed to worst case upper bounds on the number of pieces. Our approach involves two novel ingredients -- an output-sensitive algorithm for enumerating polytopes induced by a set of hyperplanes using tools from computational geometry, and an execution graph which compactly represents all the states the algorithm could attain for all possible parameter values. We illustrate our techniques by giving algorithms for pricing problems, linkage-based clustering and dynamic-programming based sequence alignment.