Entropy and sampling numbers of classes of ridge functions

TL;DR

This paper investigates the approximation and sampling complexities of high-dimensional ridge function classes, revealing their entropy properties and the impact of additional smoothness assumptions on sampling difficulty.

Contribution

It provides a detailed analysis of entropy and sampling numbers for ridge functions, highlighting the contrast between entropy compactness and sampling complexity, and introduces new tractability insights.

Findings

Entropy numbers show ridge classes are as compact as univariate Lipschitz functions.

Sampling ridge functions on the Euclidean ball faces the curse of dimensionality.

Additional smoothness assumptions can drastically reduce sampling complexity.

Abstract

We study properties of ridge functions $f(x)=g(a\cdot x)$ in high dimensions $d$ from the viewpoint of approximation theory. The considered function classes consist of ridge functions such that the profile $g$ is a member of a univariate Lipschitz class with smoothness $\alpha > 0$ (including infinite smoothness), and the ridge direction $a$ has $p$-norm $\|a\|_p \leq 1$. First, we investigate entropy numbers in order to quantify the compactness of these ridge function classes in $L_{\infty}$. We show that they are essentially as compact as the class of univariate Lipschitz functions. Second, we examine sampling numbers and face two extreme cases. In case $p=2$, sampling ridge functions on the Euclidean unit ball faces the curse of dimensionality. It is thus as difficult as sampling general multivariate Lipschitz functions, a result in sharp contrast to the result on entropy numbers. When we additionally assume that all feasible profiles have a first derivative uniformly bounded away from zero in the origin, then the complexity of sampling ridge functions reduces drastically to the complexity of sampling univariate Lipschitz functions. In between, the sampling problem's degree of difficulty varies, depending on the values of $\alpha$ and $p$. Surprisingly, we see almost the entire hierarchy of tractability levels as introduced in the recent monographs by Novak and Wo\'zniakowski.