Randomized Learning of the Second-Moment Matrix of a Smooth Function

TL;DR

This paper introduces a simple randomized algorithm to estimate the second-moment matrix of a smooth function using only point evaluations, with theoretical guarantees, aiding in active subspace identification and ridge approximation.

Contribution

It proposes a novel, assumption-free randomized method for estimating the second-moment matrix of a smooth function from point evaluations, with proven theoretical guarantees.

Findings

Algorithm effectively estimates the second-moment matrix.

Theoretical guarantees are established for the estimation accuracy.

Applicable to active subspace and ridge approximation tasks.

Abstract

Consider an open set $\mathbb{D}\subseteq\mathbb{R}^n$, equipped with a probability measure $\mu$. An important characteristic of a smooth function $f:\mathbb{D}\rightarrow\mathbb{R}$ is its \emph{second-moment matrix} $\Sigma_{\mu}:=\int \nabla f(x) \nabla f(x)^* \mu(dx) \in\mathbb{R}^{n\times n}$, where $\nabla f(x)\in\mathbb{R}^n$ is the gradient of $f(\cdot)$ at $x\in\mathbb{D}$ and $*$ stands for transpose. For instance, the span of the leading $r$ eigenvectors of $\Sigma_{\mu}$ forms an \emph{active subspace} of $f(\cdot)$, which contains the directions along which $f(\cdot)$ changes the most and is of particular interest in \emph{ridge approximation}. In this work, we propose a simple algorithm for estimating $\Sigma_{\mu}$ from random point evaluations of $f(\cdot)$ \emph{without} imposing any structural assumptions on $\Sigma_{\mu}$. Theoretical guarantees for this algorithm are established with the aid of the same technical tools that have proved valuable in the context of covariance matrix estimation from partial measurements.