An Asymptotically Optimal Multi-Armed Bandit Algorithm and Hyperparameter Optimization

TL;DR

This paper introduces BOSS, a hyperparameter optimization method combining a new bandit algorithm called SS with Bayesian Optimization, demonstrating superior performance in various deep learning model selection tasks.

Contribution

The paper proposes SS, an asymptotically optimal bandit algorithm for hyperparameter evaluation, and integrates it with Bayesian Optimization to create BOSS, a novel hyperparameter search method.

Findings

BOSS outperforms existing hyperparameter optimization methods.

SS is theoretically proven to be optimal under cumulative regret.

Empirical results show BOSS's effectiveness in NAS, DA, OD, and RL.

Abstract

The evaluation of hyperparameters, neural architectures, or data augmentation policies becomes a critical model selection problem in advanced deep learning with a large hyperparameter search space. In this paper, we propose an efficient and robust bandit-based algorithm called Sub-Sampling (SS) in the scenario of hyperparameter search evaluation. It evaluates the potential of hyperparameters by the sub-samples of observations and is theoretically proved to be optimal under the criterion of cumulative regret. We further combine SS with Bayesian Optimization and develop a novel hyperparameter optimization algorithm called BOSS. Empirical studies validate our theoretical arguments of SS and demonstrate the superior performance of BOSS on a number of applications, including Neural Architecture Search (NAS), Data Augmentation (DA), Object Detection (OD), and Reinforcement Learning (RL).