High Dimensional Level Set Estimation with Bayesian Neural Network

TL;DR

This paper introduces Bayesian Neural Network-based methods for high-dimensional Level Set Estimation, addressing scalability issues and providing theoretical analysis and practical tuning strategies, with improved results over existing approaches.

Contribution

The paper develops novel Bayesian Neural Network methods for high-dimensional LSE, including theoretical acquisition functions and hyper-parameter tuning strategies, enhancing scalability and accuracy.

Findings

Outperforms existing methods on synthetic datasets

Effective in high-dimensional settings

Provides theoretical analysis of acquisition functions

Abstract

Level Set Estimation (LSE) is an important problem with applications in various fields such as material design, biotechnology, machine operational testing, etc. Existing techniques suffer from the scalability issue, that is, these methods do not work well with high dimensional inputs. This paper proposes novel methods to solve the high dimensional LSE problems using Bayesian Neural Networks. In particular, we consider two types of LSE problems: (1) \textit{explicit} LSE problem where the threshold level is a fixed user-specified value, and, (2) \textit{implicit} LSE problem where the threshold level is defined as a percentage of the (unknown) maximum of the objective function. For each problem, we derive the corresponding theoretic information based acquisition function to sample the data points so as to maximally increase the level set accuracy. Furthermore, we also analyse the theoretical time complexity of our proposed acquisition functions, and suggest a practical methodology to efficiently tune the network hyper-parameters to achieve high model accuracy. Numerical experiments on both synthetic and real-world datasets show that our proposed method can achieve better results compared to existing state-of-the-art approaches.