Two-stage Design for Failure Probability Estimation with Gaussian Process Surrogates

TL;DR

This paper introduces a two-stage surrogate modeling approach using Gaussian processes to efficiently estimate small failure probabilities in expensive simulations, outperforming existing methods by balancing exploration and exploitation.

Contribution

A novel two-stage design strategy for Gaussian process surrogates that improves failure probability estimation with limited computational budgets.

Findings

Outperforms existing methods like exhaustive contour location and importance sampling.

Effectively estimates small failure probabilities with only hundreds of evaluations.

Demonstrated on fluid flow simulation around an airfoil.

Abstract

We tackle the problem of quantifying failure probabilities for expensive deterministic computer experiments with stochastic inputs under a fixed budget. The computational cost of the computer simulation prohibits direct Monte Carlo (MC) and necessitates a surrogate model, which may facilitate either a "surrogate MC" estimator or a surrogate-informed importance sampling estimator. We embrace the former, finding importance sampling too variable when budgets are limited, and propose a novel design strategy to effectively train a surrogate for the purpose of failure probability estimation. Existing works exhaust the entire evaluation budget on active learning through sequential contour location (CL), attempting to balance exploration with exploitation of the failure contour throughout the design, but we find exhaustive CL to be suboptimal. Instead we propose a novel two-stage surrogate design. In Stage 1, we conduct sequential CL to locate the failure contour. In Stage 2, once surrogate learning has saturated, we use a solely exploitative strategy -- allocating the remaining evaluation budget to MC samples with the highest classification entropy to ensure they are classified correctly. We propose a stopping criterion to determine the transition between stages without any tuning. Our two-stage design outperforms alternatives, including exhaustive CL and surrogate-informed importance sampling, on a variety of benchmark exercises. With these tools, we are able to effectively estimate small failure probabilities with only hundreds of simulator evaluations, showcasing functionality with both shallow and deep Gaussian process surrogates, and deploying our method on a simulation of fluid flow around an airfoil.