Discovering Active Subspaces for High-Dimensional Computer Models

TL;DR

This paper advances active subspace methods for high-dimensional models by deriving closed-form calculations for tensor product models, enabling efficient dimension reduction with various regression surrogates like MARS.

Contribution

It generalizes active subspace calculations to models expressed as linear combinations of tensor products, improving scalability and applicability beyond Gaussian process surrogates.

Findings

Closed-form active subspace calculations for tensor product models

MARS surrogate improves high-dimensional active subspace estimation

Real-world example with 240 inputs completed in under half an hour

Abstract

Dimension reduction techniques have long been an important topic in statistics, and active subspaces (AS) have received much attention this past decade in the computer experiments literature. The most common approach towards estimating the AS is to use Monte Carlo with numerical gradient evaluation. While sensible in some settings, this approach has obvious drawbacks. Recent research has demonstrated that active subspace calculations can be obtained in closed form, conditional on a Gaussian process (GP) surrogate, which can be limiting in high-dimensional settings for computational reasons. In this paper, we produce the relevant calculations for a more general case when the model of interest is a linear combination of tensor products. These general equations can be applied to the GP, recovering previous results as a special case, or applied to the models constructed by other regression techniques including multivariate adaptive regression splines (MARS). Using a MARS surrogate has many advantages including improved scaling, better estimation of active subspaces in high dimensions and the ability to handle a large number of prior distributions in closed form. In one real-world example, we obtain the active subspace of a radiation-transport code with 240 inputs and 9,372 model runs in under half an hour.