Frugal Algorithm Selection
Erdem Ku\c{s}, \"Ozg\"ur Akg\"un, Nguyen Dang, Ian Miguel

TL;DR
This paper investigates methods to reduce the training cost of automated algorithm selection by choosing optimal training subsets and employing active learning, timeout predictors, and progressive data collection, demonstrating significant cost reductions.
Contribution
It introduces novel strategies for selecting training instances in algorithm selection, including active learning and timeout-based data collection, to lower training costs.
Findings
Active learning reduces training data labeling costs.
Timeout predictors improve algorithm performance estimation.
Progressive timeout data collection further decreases training effort.
Abstract
When solving decision and optimisation problems, many competing algorithms (model and solver choices) have complementary strengths. Typically, there is no single algorithm that works well for all instances of a problem. Automated algorithm selection has been shown to work very well for choosing a suitable algorithm for a given instance. However, the cost of training can be prohibitively large due to running candidate algorithms on a representative set of training instances. In this work, we explore reducing this cost by choosing a subset of the training instances on which to train. We approach this problem in three ways: using active learning to decide based on prediction uncertainty, augmenting the algorithm predictors with a timeout predictor, and collecting training data using a progressively increasing timeout. We evaluate combinations of these approaches on six datasets from ASLib…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInnovation and Socioeconomic Development · Artificial Immune Systems Applications · Evolutionary Algorithms and Applications
MethodsSparse Evolutionary Training
