Hyper-parameter optimization based on soft actor critic and hierarchical mixture regularization

TL;DR

This paper introduces a reinforcement learning approach using soft actor critic and hierarchical mixture regularization to efficiently optimize hyper-parameters, outperforming traditional methods in speed and quality.

Contribution

It proposes a novel hyper-parameter optimization method modeled as a Markov decision process, utilizing advanced reinforcement learning techniques for improved results.

Findings

Achieves better hyper-parameters faster than existing methods

Demonstrates superior performance in experiments

Reduces optimization time significantly

Abstract

Hyper-parameter optimization is a crucial problem in machine learning as it aims to achieve the state-of-the-art performance in any model. Great efforts have been made in this field, such as random search, grid search, Bayesian optimization. In this paper, we model hyper-parameter optimization process as a Markov decision process, and tackle it with reinforcement learning. A novel hyper-parameter optimization method based on soft actor critic and hierarchical mixture regularization has been proposed. Experiments show that the proposed method can obtain better hyper-parameters in a shorter time.