Efficient optimization of expensive black-box simulators via marginal means, with application to neutrino detector design

TL;DR

This paper introduces BOMM, a novel black-box optimization method that leverages marginal mean functions to efficiently optimize high-dimensional, expensive simulators, demonstrated through neutrino detector design.

Contribution

The paper proposes BOMM, a new estimator for global optimization that outperforms traditional pick-the-winner methods, especially in high-dimensional settings, with theoretical guarantees and practical implementation.

Findings

BOMM achieves better optimization performance than existing methods.

The estimator is consistent and mitigates the curse of dimensionality.

Effective in real-world neutrino detector design applications.

Abstract

With advances in scientific computing, computer experiments are increasingly used for optimizing complex systems. However, for modern applications, e.g., the optimization of nuclear physics detectors, each experiment run can require hundreds of CPU hours, making the optimization of its black-box simulator over a high-dimensional space a challenging task. Given limited runs at inputs $\mathbf{x}_1, \cdots, \mathbf{x}_n$, the best solution from these evaluated inputs can be far from optimal, particularly as dimensionality increases. Existing black-box methods, however, largely employ this ''pick-the-winner'' (PW) solution, which leads to mediocre optimization performance. To address this, we propose a new Black-box Optimization via Marginal Means (BOMM) approach. The key idea is a new estimator of a global optimizer $\mathbf{x}^*$ that leverages the so-called marginal mean functions, which can be efficiently inferred with limited runs in high dimensions. Unlike PW, this estimator can select solutions beyond evaluated inputs for improved optimization performance. Assuming the objective function follows a generalized additive model with unknown link function and under mild conditions, we prove that the BOMM estimator not only is consistent for optimization, but also has an optimization rate that tempers the ''curse-of-dimensionality'' faced by existing methods, thus enabling better performance as dimensionality increases. We present a practical framework for implementing BOMM using the transformed additive Gaussian process surrogate model. Finally, we demonstrate the effectiveness of BOMM in numerical experiments and an application on neutrino detector optimization in nuclear physics.