FastBO: Fast HPO and NAS with Adaptive Fidelity Identification
Jiantong Jiang, Ajmal Mian

TL;DR
FastBO introduces an adaptive multi-fidelity Bayesian optimization approach for hyperparameter tuning and neural architecture search, effectively selecting fidelities for configurations to improve efficiency and performance.
Contribution
It proposes a novel adaptive fidelity identification strategy that extends single-fidelity methods to multi-fidelity settings, enhancing HPO and NAS efficiency.
Findings
Achieves strong performance in HPO and NAS tasks.
Provides a general strategy applicable to existing single-fidelity methods.
Demonstrates efficiency gains through adaptive fidelity selection.
Abstract
Hyperparameter optimization (HPO) and neural architecture search (NAS) are powerful in attaining state-of-the-art machine learning models, with Bayesian optimization (BO) standing out as a mainstream method. Extending BO into the multi-fidelity setting has been an emerging research topic, but faces the challenge of determining an appropriate fidelity for each hyperparameter configuration to fit the surrogate model. To tackle the challenge, we propose a multi-fidelity BO method named FastBO, which adaptively decides the fidelity for each configuration and efficiently offers strong performance. The advantages are achieved based on the novel concepts of efficient point and saturation point for each configuration.We also show that our adaptive fidelity identification strategy provides a way to extend any single-fidelity method to the multi-fidelity setting, highlighting its generality and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAcoustic Wave Resonator Technologies
