Automatic Hyper-Parameter Optimization Based on Mapping Discovery from Data to Hyper-Parameters

TL;DR

This paper introduces an efficient autoHPO method that learns a data-to-hyper-parameters mapping using a novel network structure, significantly improving optimization speed and effectiveness across various problems.

Contribution

The paper proposes a new network-based approach to automatically learn hyper-parameter mappings from data, enhancing autoHPO efficiency and generality.

Findings

Outperforms state-of-the-art autoHPO methods

Reduces hyper-parameter tuning time

Achieves higher optimization accuracy

Abstract

Machine learning algorithms have made remarkable achievements in the field of artificial intelligence. However, most machine learning algorithms are sensitive to the hyper-parameters. Manually optimizing the hyper-parameters is a common method of hyper-parameter tuning. However, it is costly and empirically dependent. Automatic hyper-parameter optimization (autoHPO) is favored due to its effectiveness. However, current autoHPO methods are usually only effective for a certain type of problems, and the time cost is high. In this paper, we propose an efficient automatic parameter optimization approach, which is based on the mapping from data to the corresponding hyper-parameters. To describe such mapping, we propose a sophisticated network structure. To obtain such mapping, we develop effective network constrution algorithms. We also design strategy to optimize the result futher during the application of the mapping. Extensive experimental results demonstrate that the proposed approaches outperform the state-of-the-art apporaches significantly.