ChaCha for Online AutoML
Qingyun Wu, Chi Wang, John Langford, Paul Mineiro, Marco Rossi
TL;DR
ChaCha is an online algorithm for hyperparameter tuning in online learning, dynamically selecting the best configuration over time with theoretical guarantees and demonstrated empirical effectiveness.
Contribution
Introduces the ChaCha algorithm for online hyperparameter selection with regret guarantees and practical performance across diverse datasets.
Findings
ChaCha achieves sublinear regret after including the optimal configuration.
It performs well across various datasets for featurization and hyperparameter optimization.
Provides a systematic approach to online hyperparameter tuning with theoretical and empirical validation.
Abstract
We propose the ChaCha (Champion-Challengers) algorithm for making an online choice of hyperparameters in online learning settings. ChaCha handles the process of determining a champion and scheduling a set of `live' challengers over time based on sample complexity bounds. It is guaranteed to have sublinear regret after the optimal configuration is added into consideration by an application-dependent oracle based on the champions. Empirically, we show that ChaCha provides good performance across a wide array of datasets when optimizing over featurization and hyperparameter decisions.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsMachine Learning and Data Classification · Advanced Bandit Algorithms Research · Machine Learning and Algorithms