Extending classical surrogate modelling to high-dimensions through supervised dimensionality reduction: a data-driven approach

TL;DR

This paper introduces a novel, model-agnostic approach that combines supervised dimensionality reduction with surrogate modelling techniques, enabling efficient high-dimensional uncertainty quantification up to 10,000 dimensions.

Contribution

It presents a new data-driven method that extends classical surrogate models to high dimensions by integrating supervised dimensionality reduction, overcoming the curse of dimensionality.

Findings

The approach successfully models high-dimensional problems up to 10,000 dimensions.

Combining Kriging and polynomial chaos with kernel PCA improves surrogate accuracy.

The method outperforms classical sequential dimensionality reduction and surrogate modelling on benchmarks.

Abstract

Thanks to their versatility, ease of deployment and high-performance, surrogate models have become staple tools in the arsenal of uncertainty quantification (UQ). From local interpolants to global spectral decompositions, surrogates are characterised by their ability to efficiently emulate complex computational models based on a small set of model runs used for training. An inherent limitation of many surrogate models is their susceptibility to the curse of dimensionality, which traditionally limits their applicability to a maximum of $\mathcal{O}(10^2)$ input dimensions. We present a novel approach at high-dimensional surrogate modelling that is model-, dimensionality reduction- and surrogate model- agnostic (black box), and can enable the solution of high dimensional (i.e. up to $\mathcal{O}(10^4)$) problems. After introducing the general algorithm, we demonstrate its performance by combining Kriging and polynomial chaos expansions surrogates and kernel principal component analysis. In particular, we compare the generalisation performance that the resulting surrogates achieve to the classical sequential application of dimensionality reduction followed by surrogate modelling on several benchmark applications, comprising an analytical function and two engineering applications of increasing dimensionality and complexity.