An adaptive kriging method for solving nonlinear inverse statistical problems

TL;DR

This paper introduces an adaptive kriging-based approach for efficiently solving nonlinear inverse statistical problems, especially when computational models are expensive, by optimizing experimental design to improve Bayesian inference accuracy.

Contribution

It proposes two novel heuristics, WIMSE and ECD, for sequentially designing experiments that enhance posterior estimation with limited model evaluations.

Findings

ECD criterion outperforms WIMSE in convergence speed

Adaptive designs significantly outperform classical LHD methods

Method demonstrates practical feasibility in hydraulic engineering case-study

Abstract

In various industrial contexts, estimating the distribution of unobserved random vectors Xi from some noisy indirect observations H(Xi) + Ui is required. If the relation between Xi and the quantity H(Xi), measured with the error Ui, is implemented by a CPU-consuming computer model H, a major practical difficulty is to perform the statistical inference with a relatively small number of runs of H. Following Fu et al. (2014), a Bayesian statistical framework is considered to make use of possible prior knowledge on the parameters of the distribution of the Xi, which is assumed Gaussian. Moreover, a Markov Chain Monte Carlo (MCMC) algorithm is carried out to estimate their posterior distribution by replacing H by a kriging metamodel build from a limited number of simulated experiments. Two heuristics, involving two different criteria to be optimized, are proposed to sequentially design these computer experiments in the limits of a given computational budget. The first criterion is a Weighted Integrated Mean Square Error (WIMSE). The second one, called Expected Conditional Divergence (ECD), developed in the spirit of the Stepwise Uncertainty Reduction (SUR) criterion, is based on the discrepancy between two consecutive approximations of the target posterior distribution. Several numerical comparisons conducted over a toy example then a motivating real case-study show that such adaptive designs can significantly outperform the classical choice of a maximin Latin Hypercube Design (LHD) of experiments. Dealing with a major concern in hydraulic engineering, a particular emphasis is placed upon the prior elicitation of the case-study, highlighting the overall feasibility of the methodology. Faster convergences and manageability considerations lead to recommend the use of the ECD criterion in practical applications.