Dimension Reduction via Gaussian Ridge Functions

TL;DR

This paper introduces a new Gaussian ridge function approach for dimension reduction, combining statistical and approximation techniques, with an iterative algorithm to identify subspaces, demonstrated on analytical and real-world problems.

Contribution

The paper presents a novel algorithm for Gaussian ridge functions that optimizes subspaces on the Stiefel manifold, bridging regression and approximation methods.

Findings

Near exact ridge recovery on analytical functions

Effective dimension reduction in turbomachinery case study

Posterior variance as a heuristic for subspace suitability

Abstract

Ridge functions have recently emerged as a powerful set of ideas for subspace-based dimension reduction. In this paper we begin by drawing parallels between ridge subspaces, sufficient dimension reduction and active subspaces, contrasting between techniques rooted in statistical regression and those rooted in approximation theory. This sets the stage for our new algorithm that approximates what we call a Gaussian ridge function---the posterior mean of a Gaussian process on a dimension-reducing subspace---suitable for both regression and approximation problems. To compute this subspace we develop an iterative algorithm that optimizes over the Stiefel manifold to compute the subspace, followed by an optimization of the hyperparameters of the Gaussian process. We demonstrate the utility of the algorithm on two analytical functions, where we obtain near exact ridge recovery, and a turbomachinery case study, where we compare the efficacy of our approach with three well-known sufficient dimension reduction methods: SIR, SAVE, CR. The comparisons motivate the use of the posterior variance as a heuristic for identifying the suitability of a dimension-reducing subspace.