Sequential Design of Computer Experiments with Quantitative and Qualitative Factors in Applications to HPC Performance Optimization

TL;DR

This paper introduces an adaptive sequential design method combining exploitation and exploration for optimizing computer experiments with qualitative and quantitative factors, specifically applied to HPC performance tuning.

Contribution

It proposes a novel adaptive CEE method that balances exploration and exploitation using Gaussian process models for efficient optimization in mixed-factor settings.

Findings

Effective in numerical simulations

Improves HPC performance optimization

Theoretically justified adaptive design region

Abstract

Computer experiments with both qualitative and quantitative factors are widely used in many applications. Motivated by the emerging need of optimal configuration in the high-performance computing (HPC) system, this work proposes a sequential design, denoted as adaptive composite exploitation and exploration (CEE), for optimization of computer experiments with qualitative and quantitative factors. The proposed adaptive CEE method combines the predictive mean and standard deviation based on the additive Gaussian process to achieve a meaningful balance between exploitation and exploration for optimization. Moreover, the adaptiveness of the proposed sequential procedure allows the selection of next design point from the adaptive design region. Theoretical justification of the adaptive design region is provided. The performance of the proposed method is evaluated by several numerical examples in simulations. The case study of HPC performance optimization further elaborates the merits of the proposed method.