Improving Hyperparameter Optimization by Planning Ahead

TL;DR

This paper introduces a planning-based transfer learning method for hyperparameter optimization that uses model predictive control with trajectory sampling, outperforming existing algorithms on multiple datasets.

Contribution

It proposes a novel transfer learning approach within model-based reinforcement learning, employing a look-ahead strategy for hyperparameter selection to improve optimization efficiency.

Findings

Outperforms state-of-the-art HPO algorithms on three meta-datasets.

Uses an ensemble of probabilistic models for trajectory sampling.

Employs a simple look-ahead policy for hyperparameter selection.

Abstract

Hyperparameter optimization (HPO) is generally treated as a bi-level optimization problem that involves fitting a (probabilistic) surrogate model to a set of observed hyperparameter responses, e.g. validation loss, and consequently maximizing an acquisition function using a surrogate model to identify good hyperparameter candidates for evaluation. The choice of a surrogate and/or acquisition function can be further improved via knowledge transfer across related tasks. In this paper, we propose a novel transfer learning approach, defined within the context of model-based reinforcement learning, where we represent the surrogate as an ensemble of probabilistic models that allows trajectory sampling. We further propose a new variant of model predictive control which employs a simple look-ahead strategy as a policy that optimizes a sequence of actions, representing hyperparameter candidates to expedite HPO. Our experiments on three meta-datasets comparing to state-of-the-art HPO algorithms including a model-free reinforcement learning approach show that the proposed method can outperform all baselines by exploiting a simple planning-based policy.