Explicit error bounds for randomized Smolyak algorithms and an application to infinite-dimensional integration

TL;DR

This paper analyzes randomized Smolyak algorithms, providing explicit error bounds and demonstrating their effectiveness in high-dimensional and infinite-dimensional integration problems, with applications to weighted reproducing kernel Hilbert spaces.

Contribution

It introduces explicit error bounds for randomized Smolyak algorithms and applies these results to infinite-dimensional integration, highlighting when randomized methods outperform deterministic ones.

Findings

Derived upper and lower error bounds with explicit dependence on variables and evaluations.

Established convergence rates for N-th minimal errors in infinite-dimensional integration.

Characterized spaces where randomized algorithms outperform deterministic counterparts.

Abstract

Smolyak's method, also known as hyperbolic cross approximation or sparse grid method, is a powerful tool to tackle multivariate tensor product problems solely with the help of efficient algorithms for the corresponding univariate problem. In this paper we study the randomized setting, i.e., we randomize Smolyak's method. We provide upper and lower error bounds for randomized Smolyak algorithms with explicitly given dependence on the number of variables and the number of information evaluations used. The error criteria we consider are the worst-case root mean square error (the typical error criterion for randomized algorithms, often referred to as "randomized error") and the root mean square worst-case error (often referred to as "worst-case error"). Randomized Smolyak algorithms can be used as building blocks for efficient methods such as multilevel algorithms, multivariate decomposition methods or dimension-wise quadrature methods to tackle successfully high-dimensional or even infnite-dimensional problems. As an example, we provide a very general and sharp result on the convergence rate of N-th minimal errors of infnite-dimensional integration on weighted reproducing kernel Hilbert spaces. Moreover, we are able to characterize the spaces for which randomized algorithms for infnte-dimensional integration are superior to deterministic ones. We illustrate our fndings for the special instance of weighted Korobov spaces. We indicate how these results can be extended, e.g., to spaces of functions whose smooth dependence on successive variables increases ("spaces of increasing smoothness") and to the problem of L2-approximation (function recovery).

Equations

$$F_d:=F^{(1)}\otimes \cdots \otimes F^{(d)},$$

$$G_d:=G^{(1)}\otimes \cdots \otimes G^{(d)},$$

$$S_d:=S^{(1)}\otimes \cdots \otimes S^{(d)}.$$

$$A:\Omega \to L(F_d,G_d)$$

$${\mathcal{O}}^{\mathrm{ran}}:={\mathcal{O}}^{\mathrm{ran},\mathrm{lin}}(\Omega ,F_d,G_d):=\{A:\Omega \to L(F_d,G_d)|A\text{ is a randomized linear operator}\}.$$

$${\mathcal{O}}^{\det }:={\mathcal{O}}^{\det ,\mathrm{lin}}(F_d,G_d):=L(F_d,G_d)\subset {\mathcal{O}}^{\mathrm{ran},\mathrm{lin}}(\Omega ,F_d,G_d),$$

$$e^{\rm{r}}(A):=e^{\rm{r}}(S_{d},A):=\sup\limits_{\begin{subarray}{c}\lVert f\rVert_{F_{d}}\leq 1\end{subarray}}\operatorname{{\bf E}}\bigg{[}\lVert(S_{d}-A)f\rVert^{2}\bigg{]}^{\frac{1}{2}},$$

$$e^{\rm{w}}(A):=e^{\rm{w}}(S_{d},A):=\operatorname{{\bf E}}\bigg{[}\sup\limits_{\begin{subarray}{c}\lVert f\rVert_{F_{d}}\leq 1\end{subarray}}\lVert(S_{d}-A)f\rVert^{2}\bigg{]}^{\frac{1}{2}}.$$

$$e^{\mathrm{d}}(A):=e^{\mathrm{d}}(S_d,A):=\underset{\begin{array}{c}\Vert f{\Vert }_{F_d}\le 1\end{array}}{\sup }\Vert (S_d-A)f\Vert ,$$

$$S_d(f)={\int }_{E}f\mathrm{d}\mu ,f\in F_d.$$

$$S_d(f)=f,f\in F_d.$$

$${\Delta }_0^{(n)}:=U_0^{(n)}:=0,{\Delta }_l^{(n)}:=U_l^{(n)}-U_{l-1}^{(n)},l\in \mathbb{N},$$

$$Q(L,d):=\left\{l\in {\mathbb{N}}^d||l|\le L\right\}.$$

$$A(L,d)f:\Omega \to G_d,\omega \mapsto \left(\sum _{l\in Q(L,d)}\underset{n=1}{\overset{d}{⨂}}{\Delta }_{l_n}^{(n)}({\omega }_n)\right)f.$$

$$A(L,d)=\sum _{L-d+1\le |l|\le L}(-1{)}^{L-|l|}(\genfrac{}{}{0ex}{}{d-1}{L-|l|})\underset{n=1}{\overset{d}{⨂}}{U}_{l_n}^{(n)},$$

$$U_l^{(n)}f\in L^2({\Omega }^{(n)},G^{(n)})\text{ for all }f\in F^{(n)}.$$

$${\mu }_{l,n}(\omega ):=\underset{\begin{array}{c}\Vert f{\Vert }_{F^{(n)}}\le 1\end{array}}{\sup }\Vert (U_l^{(n)}f)({\omega }_n)\Vert <\infty \text{ for all }{\omega }_n\in {\Omega }^{(n)}$$

$$\Vert {\mu }_{l,n}{\Vert }_{L^2({\Omega }^{(n)},\mathbb{R})}<\infty .$$

$$\Vert S^{(n)}{\Vert }_{\mathrm{op}}\le B,$$

$$e^{\mathrm{x}}(S^{(n)},U_l^{(n)})\le C{D}^l,$$

$$\sup\limits_{\begin{subarray}{c}\lVert f\rVert_{F^{(n)}}\leq 1\end{subarray}}\operatorname{{\bf E}}\bigg{[}\lVert\underbrace{U_{l}^{(n)}f-U_{l-1}^{(n)}f}_{=\Delta^{(n)}_{l}f}\rVert^{2}\bigg{]}^{\frac{1}{2}}\leq ED^{l},$$

$$\operatorname{{\bf E}}\bigg{[}\sup\limits_{\begin{subarray}{c}\lVert f\rVert_{F^{(n)}}\leq 1\end{subarray}}\lVert\underbrace{U_{l}^{(n)}f-U_{l-1}^{(n)}f}_{=\Delta^{(n)}_{l}f}\rVert^{2}\bigg{]}^{\frac{1}{2}}\leq ED^{l}.$$

$$A:F\to L^{2}(\Omega,G),\quad f\mapsto\big{(}\omega\mapsto Af(\omega)\big{)}.$$

$$\underset{\begin{array}{c}\Vert f{\Vert }_{F^{(n)}}\le 1\end{array}}{\sup }E{\left[\Vert U_l^{(n)}f{\Vert }^2\right]}^{1/2}\le \frac{ED}{1-D},$$

$$L^2({\Omega }_1,H_1)\otimes L^2({\Omega }_2,H_2)\cong L^2({\Omega }_1\times {\Omega }_2,H_1\otimes H_2),$$

$$\Vert S_d{\Vert }_{\mathrm{op}}=\underset{\Vert f{\Vert }_{F_d}\le 1}{\sup }\Vert S_{d}f{\Vert }_{L^2(\Omega ,G)}=\underset{\Vert f{\Vert }_{F_d}\le 1}{\sup }E{\left[\Vert S_{d}f(\omega ){\Vert }^2\right]}^{1/2},$$

$$e^{\mathrm{r}}(S_d,A)=\underset{\Vert f{\Vert }_{F_d}\le 1}{\sup }\Vert (S_d-A)f{\Vert }_{L^2(\Omega ,G_d)}=\Vert S_d-A{\Vert }_{\mathrm{op}}.$$

$$S:F\to G,A:\Omega \to L(F,G)$$

$$S:F\to L^2(\Omega ,G),A:F\to L^2(\Omega ,G).$$

$$e^{\rm{x}}(S_{d},A(L,d))\leq CB^{d-1}D^{L-d+1}\sum\limits_{j=0}^{d-1}\bigg{(}\frac{ED}{B}\bigg{)}^{j}\binom{L-d+j}{j}\leq CH^{d-1}\binom{L}{d-1}D^{L},$$

Full text

In this paper we study the randomized setting, i.e., we randomize Smolyak’s method. We provide upper and lower error bounds for randomized Smolyak algorithms with explicitly given dependence on the number of variables and the number of information evaluations used. The error criteria we consider are the worst-case root mean square error (the typical error criterion for randomized algorithms, often referred to as “randomized error”) and the root mean square worst-case error (often referred to as “worst-case error”).

Randomized Smolyak algorithms can be used as building blocks for efficient methods such as multilevel algorithms, multivariate decomposition methods or dimension-wise quadrature methods to tackle successfully high-dimensional or even infinite-dimensional problems. As an example, we provide a very general and sharp result on the convergence rate of $N$-th minimal errors of infinite-dimensional integration on weighted reproducing kernel Hilbert spaces. Moreover, we are able to characterize the spaces for which randomized algorithms for infinte-dimensional integration are superior to deterministic ones. We illustrate our findings for the special instance of weighted Korobov spaces. We indicate how these results can be extended, e.g., to spaces of functions whose smooth dependence on successive variables increases (“spaces of increasing smoothness”) and to the problem of $L^{2}$-approximation (function recovery).

1 Introduction

Smolyak’s method or algorithm, also known as sparse grid method, hyperbolic cross approximation, Boolean method, combination technique or discrete blending method, was outlined by Smolyak in [56]. It is a general method to treat multivariate tensor product problems. Its major advantage is the following: to tackle a multivariate tensor product problem at hand one only has to understand the corresponding univariate problem. More precisely, Smolyak’s algorithm uses algorithms for the corresponding univariate problem as building blocks, and it is fully determined by the choice of those algorithms. If those algorithms for the univariate problem are optimal, then typically Smolyak’s algorithm for the multivariate problem is almost optimal, i.e., its convergence rate is optimal up to logarithmic factors.

Today Smolyak’s method is widely used in scientific computing and there exists a huge number of scientific articles dealing with applications and modifications of it. A partial list of papers (which is, of course, very far from being complete) on deterministic Smolyak algorithms may contain, e.g., the articles [64, 65] for general approximation problems, [16, 6, 58, 3, 13, 46, 17, 18, 51, 26, 31] for numerical integration, [28, 5, 57, 59, 53, 62, 11] for function recovery, and [50, 70, 45, 68, 69, 14, 15] for other applications. Additional references and further information may be found in the survey articles [4, 29], the book chapters [47, Chapter 15], [60, Chapter 4], and the books [7, 12].

On randomized Smolyak algorithms much less is known. Actually, we are only aware of two articles that deal with randomized versions of Smolyak’s method, namely [10] and [34]. In [10] Dick et al. investigate a specific instance of the randomized Smolyak method and use it as a tool to show that higher order nets may be used to construct integration algorithms achieving almost optimal order of convergence (up to logarithmic factors) of the worst case error in certain Sobolev spaces. In [34] Heinrich and Milla employ the randomized Smolyak method as a building block of an algorithm to compute antiderivatives of functions from $L^{p}([0,1]^{d}),$ allowing for fast computation of antiderivative values for any point in $[0,1]^{d}.$ Note that in both cases the randomized Smolyak method is applied as an ad hoc device and none of the papers gives a systematic treatment of its properties.

With this paper we want to start a systematic treatment of randomized Smolyak algorithms. Similar to the paper [64], where deterministic Smolyak methods were studied, we discuss the randomized Smolyak method for general linear approximation problems on tensor products of Hilbert spaces. Examples of such approximation problems are numerical integration or $L^{2}$-approximation, i.e., function recovery.

The error criteria for randomized algorithms or, more generally, randomized operators that we consider are extensions of the worst-case error for deterministic algorithms. The first error criterion is the worst-case root mean square error, often referred to as “randomized error”. This error criterion is typically used to assess the quality of randomized algorithms. The second error criterion is the root mean square worst-case error, often referred to as “worst-case error”. This quantity is commonly used to prove the existence of a good deterministic algorithm with the help of the “pidgeon hole principle”: It arises as an average of the usual deterministic worst case error over a set of deterministic algorithms $\mathcal{A}$ endowed with a probability measure $\mu$. If the average is small, there exists at least one algorithm in $\mathcal{A}$ with small worst-case error, see, e.g., [10] or [55]. Notice that the pair $(\mathcal{A},\mu)$ can be canonically identified with a randomized algorithm.

We derive upper error bounds for both error criteria for randomized Smolyak algorithms with explicitly given dependence on the number of variables and the number of information evaluations used. The former number is the underlying dimension of the problem, the latter number is typically proportional to the cost of the algorithm. The upper error bounds show that the randomized Smolyak method can be efficiently used at least in moderately high dimension. We complement this result by providing lower error bounds for randomized Smolyak algorithms that nearly match our upper bounds.

As in the deterministic case, our upper and lower error bounds contain logarithmic factors whose powers depend linearly on the underlying dimension $d$, indicating that the direct use of the randomized Smolyak method in very high dimension may be prohibitive. Nevertheless, our upper error bounds shows that randomized Smolyak algorithms make perfect building blocks for more sophisticated algorithms such as multilevel algorithms (see, e.g., [33, 35, 19, 20, 21, 36, 43, 22, 2, 8, 39]), multivariate decomposition methods (see, e.g., [40, 52, 63, 23, 8, 9]) or dimension-wise quadrature methods (see [30]). We demonstrate this in the case of the infinite-dimensional integration problem on weighted tensor products of reproducing kernel Hilbert spaces with general kernels. We provide the exact polynomial convergence rate of $N$-th minimal errors—the corresponding upper error bound is established by multivariate decomposition methods based on randomized Smolyak algorithms.

The paper is organized as follows: In Section 2 we provide the general multivariate problem formulation and illustrate it with two examples. In Section 3 we introduce the randomized multivariate Smolyak method building on randomized univariate algorithms. Our assumptions about the univariate randomized algorithms resemble the ones made in [64] in the deterministic case. In Remark 2 we observe that we may identify our randomized linear approximation problem of interest with a corresponding deterministic $L^{2}$-approximation problem.

In Section 4 we follow the course of [64] and establish first error bounds in terms of the underlying dimension of the problem and the level of the considered Smolyak algorithm, see Theorem 3 and Corollary 4. For the randomized error criterion Remark 2 helps us to boil the error analysis of the randomized Smolyak method down to the error analysis of the deterministic Smolyak method provided in [64]. For the worst case error criterion Remark 2 is of no help and therefore we state the details of the analysis.

Up to this point we consider general randomized operators to approximate the solution we are seeking for. In Section 5 we focus on randomized algorithms and the information evalutions they use. In Theorem 18 we present upper error bounds for the randomized Smolyak method where the dependence on the underlying dimension of the problem and on the number of information evalutions is revealed. In Corollary 14 we provide lower error bounds for the randomized Smolyak method.

In Section 6 we apply our upper error bounds for randomized Smolyak algorithms to the infinite integration problem. After introducing the setting, we provide the exact polynomial convergence rate of $N$-th minimal errors in Theorem 18. In Corollary 19 we compare the power of randomized algorithms and deterministic algorithms for infinite-dimensional integration, and in Corollary 20 we illustrate the result of Theorem 18 for weighted Korobov spaces. In Remark 21 and Remark 22 we discuss previous contributions to the considered infinite integration problem and generalizations to other settings such as to function spaces with increasing smoothness or to the $L^{2}$-approximation problem.

In the appendix we provide for the convenience of the reader a self-contained proof of a folklore result on the convergence rates of randomized algorithms on Korobov spaces.

2 Formulation of the Problem

Let $d\in\mathbb{N}$. For $n=1,\ldots,d,$ let $F^{(n)}$ be a separable Hilbert space of real valued functions, $G^{(n)}$ be a separable Hilbert space, and $S^{(n)}:F^{(n)}\to G^{(n)}$ be a continuous linear operator. We consider now the tensor product spaces $F_{d}$ and $G_{d}$ given by

[TABLE]

[TABLE]

and the tensor product operator $S_{d}$ (also called solution operator) given by

[TABLE]

We frequently use results concerning tensor products of Hilbert spaces and tensor product operators without giving explicit reference, for details on this subject see, e.g., [67]. We denote the norms in $F^{(n)}$ and $F_{d}$ by $\|\cdot\|_{F^{(n)}}$ and $\|\cdot\|_{F_{d}}$ respectively, and the norms in $G^{(n)}$ and $G_{d}$ simply by $\|\cdot\|$. Furthermore, $L(F_{d},G_{d})$ denotes the space of all bounded linear operators between $F_{d}$ and $G_{d}.$

$S_{d}(f)$ may be approximated by randomized linear algorithms or, more generally, by randomized linear operators. We define a randomized linear operator $A$ to be a mapping

[TABLE]

such that $Af:\Omega\rightarrow G_{d}$ is a random variable for each $f\in F_{d}$; here $(\Omega,\Sigma,\operatorname{{\bf P}})$ is some probability space and $G_{d}$ is endowed with its Borel $\sigma-$field. We put

[TABLE]

Obviously one may interpret deterministic bounded linear operators as randomized linear operators with trivial dependance on $\Omega$. Accordingly, we put

[TABLE]

where the inclusion is based on the identification of $A\in L(F_{d},G_{d})$ with the constant mapping $\Omega\ni\omega\mapsto A$.

A *(randomized) linear approximation problem * is given by a quadruple $\{S_{d},F_{d},G_{d},\mathcal{O}(\Omega)\},$ where $\mathcal{O}(\Omega)\subseteq\mathcal{O}^{\operatorname{ran},\operatorname{lin}}(\Omega,F_{d},G_{d})$ denotes the class of admissible randomized linear operators. We are mainly interested in results for randomized linear algorithms, which constitute a subclass of $\mathcal{O}^{\operatorname{ran}}$ and will be introduced in Section 5.

Consider a randomized linear operator $A$ meant to approximate $S_{d}$. The randomized error of the operator is given by

[TABLE]

and the* (root mean square) worst case error* is

[TABLE]

Clearly we have $0\leq e^{\rm{r}}(S_{d},A)\leq e^{\rm{w}}(S_{d},A)$.

Notice that for a deterministic linear Operator $A$ both errors coincide with the deterministic worst case error

[TABLE]

i.e., $e^{\rm d}(S_{d},A)=e^{\rm r}(S_{d},A)=e^{\rm w}(S_{d},A)$.

We finish this section by giving two examples of typical tensor product problems that fit into the framework given above.

Example 1**.**

For $n=1,\ldots,d$ let $D^{(n)}\neq\emptyset$ be an arbitrary domain and let $\rho^{(n)}$ be a positive measure on $D^{(n)}$. Denote by $E$ the Cartesian product $D^{(1)}\times\cdots\times D^{(d)}$ and by $\mu$ the product measure $\otimes_{n=1}^{d}\rho$ on $E$.

• (i)

By choosing $F^{(n)}\subset L^{2}(D^{(n)},\rho^{(n)})$, $G^{(n)}:=\mathbb{R}$, and $S^{(n)}$ to be the integration functional $S^{(n)}(f)=\int_{D^{(n)}}f\,{\rm d}\rho^{(n)}$, we obtain $F_{d}\subset L^{2}(E,\mu)$, $G_{d}=\mathbb{R}$, and $S_{d}$ is the integration functional on $F_{d}$ given by

[TABLE]

The integration problem is now to compute or approximate for given $f\in F_{d}$ the integral $S_{d}(f)$.

• (ii)

By choosing $F^{(n)}\subset G^{(n)}:=L^{2}(D^{(n}),\rho^{(n)})$ and $S^{(n)}$ to be the embedding operator from $F^{(n)}$ into $G^{(n)}$, we obtain $F_{d}\subset G_{d}=L^{2}(E,\mu)$ and $S_{d}$ is the embedding operator from $F_{d}$ into $G_{d}$ given by

[TABLE]

The $L^{2}$-approximation problem is now to reconstruct a given function $f\in F_{d}$, i.e., to compute or approximate $S_{d}(f)$; the reconstruction error is measured in the $L^{2}$-norm.

Note that in both problem formulations above the phrase “a given function $f$” does not necessarilly mean that the whole function is known. Usually there is only partial information about the function available (like a finite number of values of the function or of its derivatives or a finite number of Fourier coefficients) available. We discuss this point in more detail in Section 5.1.

3 Smolyak Method for Tensor Product Problems

From now on we are interested in randomizing the Smolyak method which is to be defined in this section. Assume that for every $n=1,2,\ldots,d,$ we have a sequence of randomized linear operators $(U_{l}^{(n)})_{l\in\mathbb{N}},$ which approximate the solution operator $S^{(n)}$ such that for every $f\in F^{(n)}$ it holds: $U_{l}^{(n)}f$ is a random variable on a probability space $(\Omega^{(n)},\Sigma^{(n)},\operatorname{{\bf P}}^{(n)}).$ We shall refer to separate $U^{(n)}_{l}$ as to building blocks.

Put $\Omega:=\Omega^{(1)}\times\ldots\times\Omega^{(d)},\Sigma:=\bigotimes_{n=1}^{d}\Sigma^{(n)},\operatorname{{\bf P}}:=\bigotimes_{n=1}^{d}\operatorname{{\bf P}}^{(n)}$. We denote

[TABLE]

and

[TABLE]

Note that if $L\geq d$, then $|Q(L,d)|=\binom{L}{d}$. For $f\in F_{d}$ the randomized Smolyak method of level L approximating the tensor product problem $\{S_{d},F_{d},G_{d},\mathcal{A}(\Omega)\}$ is given by

[TABLE]

We would like to stress that due to the definition of the probability space $(\Omega,\Sigma,\operatorname{{\bf P}})$ for given $f_{n}\in F^{(n)},n=1,2,\ldots,d,$ the families $((U^{(n)}_{l}f_{n})_{l\in\mathbb{N}}),n=1,2,\ldots,d,$ are mutually independent. Note that for $L<d$ the Smolyak method is the zero operator. Therefore, we will always assume (without stating it explicitly every time) that $L\geq d.$

It can be verified that the following representation holds

[TABLE]

cf. [64, Lemma 1]. When investigating the randomized error we need that for every $l\in\mathbb{N}$ and $n=1,\ldots,d$

[TABLE]

In the worst case error analysis we require for every $l\in\mathbb{N}$ and $n=1,\ldots,d$

[TABLE]

and that $\mu_{l,n}:\Omega\rightarrow[0,\infty)$ is measurable with

[TABLE]

Let $\rm{x}\in\{\rm{r},\rm{w}\}.$ When considering the error $e^{\rm{x}}(S_{d},A(L,d)),$ we assume that there exist constants $B,C,E>0$ and $D\in(0,1)$ such that for every $n=1,2,\ldots,d,$ and every $l\in\mathbb{N}$

[TABLE]

[TABLE]

and additionally in the randomized setting

[TABLE]

and in the worst case setting

[TABLE]

Note that (9) implies the conditions (10) and (11) with a constant $E:=C(1+D^{-1})$ for all $l\geq 2.$ Still (10) and (11) may even hold for some smaller $E.$

Remark 2**.**

For our randomized error analysis it would be convenient to identify a randomized linear operator $A:\Omega\rightarrow L(F,G),$ $F,G$ separable Hilbert spaces, with the mapping (12) which we again denote by $A:$

[TABLE]

We now show that this identification makes sense for all the operators we are considering. We start with the building blocks $U^{(n)}_{l}.$ From (10) we obtain

[TABLE]

implying $(U^{(n)}_{l}f)(\cdot)\in L^{2}(\Omega^{(n)},G^{(n)})$ for all $f\in F^{(n)}.$ The building blocks $U_{l}^{(n)}$ are obviously linear as mappings $F^{(n)}\rightarrow L^{2}(\Omega^{(n)},G^{(n)})$ and, due to (13), also bounded, i.e. continuous. Now, since for arbitrary sample spaces $\Omega_{1},\Omega_{2}$ and separable Hilbert spaces $H_{1},H_{2}$ it holds

[TABLE]

we have that $(\bigotimes_{n=1}^{d}U^{(n)}_{l_{n}})(f)(\cdot)$ lies in $L^{2}(\Omega,G_{d})$ for $f\in F_{d}.$ Clearly, the tensor product operator $\bigotimes_{n=1}^{d}U^{(n)}_{l_{n}}$ is a bounded linear mapping $F\to L^{2}(\Omega,G_{d})$. Since due to (4) the Smolyak method $A(L,d)$ may be represented as a finite sum of such tensor product operators, it is also a bounded linear map $F\to L^{2}(\Omega,G_{d}).$

If we formally consider $S_{d}$ as an operator $F_{d}\to L^{2}(\Omega,G_{d})$, f\mapsto\big{(}\omega\mapsto S_{d}f\big{)} (i.e., an operator that maps into the constant $L^{2}$-functions), then $S_{d}$ is still linear and continuous with operator norm

[TABLE]

and the usual randomized error can be written as

[TABLE]

The worst case error unfortunately does not allow for a representation as operator norm similar to (14).

Note that the above identification turns a randomized approximation problem

[TABLE]

into a deterministic $L^{2}$-approximation problem

[TABLE]

4 Error Analysis in Terms of the Level

We now perform the error analysis of the approximation of $S_{d}$ by the Smolyak method $A(L,d)$ in terms of the level $L,$ which may be done under the rather general assumptions of Sections $2$ and $3.$

Theorem 3**.**

For $L,d\in\mathbb{N},L\geq d$ let $A(L,d)$ be a randomized Smolyak method as described in Section 3. Let $\rm{x}\in\{\rm{w},\rm{r}\}$. Assume (8), (9) and, dependently on the setting, for $\rm{x}=\rm{r}$ additionally assume (5),(10) and for $x=\rm{w}$ additionally assume (6), (7) and (11). Then we have

[TABLE]

where $H=\max\{\frac{B}{D},E\}.$

Proof.

The second inequality in (15) follows easily by using $\sum_{j=0}^{d-1}\binom{L-d+j}{j}=\binom{L}{d-1}$ and estimating $(\frac{ED}{B})^{j}\leq\max\{1,(\frac{ED}{B})^{d-1}\},$ so all there remains to be done is proving the first inequality.

Firstly we shall focus on the worst case error bound. Note that for a fixed $\omega\in\Omega$

[TABLE]

Now we may proceed similarly as in the proof of Lemma $2$ from [64], by induction on $d,L$ for $d\in\mathbb{N}$ and $L\in\{d,d+1,\ldots\}$ . For $d=1$ and any $L\in\mathbb{N}_{\geq d}$ we have $S_{d}=S^{(1)}$ and $A(L,1)=U^{(1)}_{L},$ so the statement is just the condition (9). Suppose we have already proved the claim for $L,d$ and want to prove it for $L+1,d+1.$ Using

[TABLE]

and Minkowski’s inequality we get

[TABLE]

We use Minkowski’s inequality, properties of tensor product operator norms, the fact that component algorithms $U^{(n)}_{l},l\in\mathbb{N}$ are randomized independently for different $n\in\{1,\ldots,d\}$, (9) and (11) to obtain

[TABLE]

Furthermore, using (8),

[TABLE]

Therefore we have

[TABLE]

and using the induction hypothesis finishes the proof for the worst case error.

Now consider the randomized error. By similar calculations as in the first part of the proof one could show that the claim holds true for the randomized error for elementary tensors. Then however, one encounters problems trying to lift it to the whole Hilbert space. The difficulty lies in the fact that the randomized error is not an operator norm of some tensor product operator, which would have enabled us to write it as a product of norms of the corresponding univariate operators and which has proved to be useful in bounding the worst case error. To get round it we need a different approach. The idea is to interpret a randomized problem as a deterministic $L^{2}-$approximation problem. As already explained in the Remark 2 we may identify $(S_{d}-A(L,d)):\Omega\rightarrow L(F_{d},G_{d})$ with an operator $F_{d}\rightarrow L^{2}(\Omega,G_{d})$ again denoted by $(S_{d}-A(L,d)).$ Then however $e^{\rm{r}}(S_{d},A)=\|S_{d}-A\|_{\operatorname{op}}$ and we may proceed exactly as in Lemma 2 from [64], which finishes the proof. ∎

We may generalize the result of the Theorem 3 by allowing for more flexibility in convergence rates in (9), (10) and (11). It can be used to capture additional logarithmic factors in the error bounds for the building blocks algorithms. This turns out to be particularly useful when investigating the error bounds for Smolyak methods whose building blocks are, e.g., multivariate quadratures or approximation algorithms, as it is the case in [10]. Suppose namely there exists a constant $D\in(0,1)$ and non decreasing sequences of positive numbers $(C_{l})_{l},(E_{l})_{l},l\in\mathbb{N},$ such that for every $l\in\mathbb{N}$

[TABLE]

Moreover, in case of the randomized error

[TABLE]

and in case of the worst case error

[TABLE]

It is now easy to prove Corollary 4 along the lines of the proof of Theorem 3.

Corollary 4**.**

For $L,d\in\mathbb{N},L\geq d$ let $A(L,d)$ be a randomized Smolyak method as described in Section 3. Let $\rm{x}\in\{\rm{w},\rm{r}\}$. Assume (8), (16) and, dependently on the setting, for $\rm{x}=\rm{r}$ assume (5),(17) and for $x=\rm{w}$ assume (6), (7) and (18). Then we have

[TABLE]

where $H_{L}=\max\{\frac{B}{D},E_{L-1}\}.$

Remark 5**.**

Note that applying Corollary 4 to the uni- or multivariate building block algorithms error bounds from [10] we may reproduce the error bounds obtained in this paper for the final (higher dimensional) Smolyak method.

5 Error Analysis in Terms of Information

5.1 Algorithms

Consider a linear approximation problem given by $\{S,F,G,\mathcal{O}(\Omega)\}.$ The aim of this section is to specify those linear operators that we want to call algorithms and to explain the typical information-based complexity framework for investigating the error of an algorithm in terms of the cardinality of information, for further reference see, e.g., [61]. To this end we shall specify a class of linear bounded functionals on $F$ called admissible information functionals and denoted by $\Lambda$, which will become one more parameter of the approximation problem. Given a constant $\tau\in\mathbb{N}_{0}$ and, if $\tau>0$, a collection of $\lambda_{i}\in\Lambda,i=1,\ldots,\tau,$ the information operator $\mathcal{N}:F\rightarrow\mathbb{R}^{\max\{\tau,1\}}$ applied to $f\in F$ is determined via

[TABLE]

Note that we are considering only non-adaptive information, meaning that the information functionals used do not depend on $f\in F.$

A deterministic linear operator $A\in\mathcal{O}^{\operatorname{det},\operatorname{lin}}(F,G)$ is called a deterministic linear algorithm if it admits a representation

[TABLE]

where $\mathcal{N}$ is an information operator and $\phi:\mathbb{R}^{\max\{{\tau},1\}}\rightarrow G$ is an arbitrary mapping. We denote the number of information functionals used by the deterministic algorithm $A$ for any input $f\in F$ by $\operatorname{card^{det}}(A,F),$ i.e.,

[TABLE]

We denote the class of deterministic linear algorithms with admissible information functionals $\Lambda$ by $\mathcal{A}^{\operatorname{det},\operatorname{lin}}(F,G,\Lambda).$

Let $(V_{l})_{l\in\mathbb{N}}$ be an arbitrary sequence of algorithms and let $(\lambda_{l,i})_{i\in[m_{l}]}$ be the information functionals used by $V_{l}.$ We say that the sequence $(V_{l})_{l}$ * uses nested information* if for every $a<b$

[TABLE]

A randomized linear algorithm $A\in\mathcal{O}^{\operatorname{ran},\operatorname{lin}}(\Omega,F,G)$ is a mapping

[TABLE]

such that $\omega\mapsto\operatorname{card^{det}}(A(\omega),F)\text{ is a random variable.}$

We denote the class of randomized linear algorithms with admissible information functionals $\Lambda$ by $\mathcal{A}^{\operatorname{ran},\operatorname{lin}}(\Omega,F,G,\Lambda)=:\mathcal{A}(\Omega,\Lambda).$

For a randomized linear algorithm $A$ we may finally define

[TABLE]

We say that the information used by a sequence of randomized linear algorithms is nested if it is nested for each $\omega\in\Omega.$ Note that the information used by $(A(L,d))_{L\geq d}$ is nested.

Now we would like to make some reasonable assumptions on the cost of building blocks of the Smolyak method. Consider a randomized Smolyak method as described in Section 3 with building blocks being randomized algorithms. Let

[TABLE]

Notice that $m_{0,n}=0.$ For $d\in\mathbb{N},L=d,d+1,\ldots$ put

[TABLE]

Let us assume that for every $n\in\{1,\ldots,d\}$ the sequence $(m_{l,n})_{l\in\mathbb{N}}$ is non-decreasing and that there exist constants $1\leq K_{\rm{low}}\leq K_{\rm{up}},1<K$ such that for every $n=1,\ldots,d,l\in\mathbb{N}$ it holds

[TABLE]

Note that this implies

[TABLE]

Example 6**.**

Consider the integration problem from Example 1. Let $s\in\mathbb{N}.$ For $n=1,\ldots,d$ let $D^{(n)}=[0,1]^{s},$ $\rho^{(n)}$ be Lebesgue measure on $[0,1]^{s}$ and $F^{(n)}$ be some reproducing kernel Hilbert space of functions defined on $[0,1]^{s}$ (e.g., a Sobolev space with sufficiently high smoothness parameter).

Choose a prime number $b\geq s$ and for $n=1,\ldots,d,$ and $l\in\mathbb{N}$ let $\mathcal{P}_{l}^{(n)}$ be a scrambled $(0,l,s)-$net in base $b$ as introduced in [48]. Now

[TABLE]

is a randomized algorithm. Moreover, if we randomize $(U^{(n)}_{l})_{n,l}$ in such a way that the families $(U^{(n)}_{l})_{l}$ are independent then we may use them as building blocks of the Smolyak method and all the results of this paper apply, cf. also [10].

5.2 Upper Error Bounds

Throughout the whole section we require that the assumptions of Theorem 3 hold. Let us define $\alpha:=\frac{\log(\frac{1}{D})}{\log(K)},$ where $K$ is as in (20) and $D$ is as in (9). We define the polynomial convergence rate of the algorithms $U^{(n)}_{l},l\in\mathbb{N}$ by

[TABLE]

where $\rm{x}\in\{\rm{r},\rm{w}\}.$ It is straightforward to verify that $\alpha\leq\mu^{(n)}_{\rm{x}}$ for every $n.$ Indeed,we have

[TABLE]

because of

[TABLE]

Hence for each $n\in\{1,\ldots,d\}$ the quantity $\alpha$ is a lower bound on the polynomial order of convergence $\mu^{(n)}_{\rm{x}}$ of the algorithms $U^{(n)}_{l},l\in\mathbb{N},$ and can be chosen arbitrarily close to $\mu^{(n)}_{\rm{x}}$ if the constants $C$ and $D$ in (9) are chosen appropriately.

The aim of this section is to develop upper bounds on the error of $d-$variate Smolyak method in terms of $N,d$ and $\alpha.$ More concretely we prove the following theorem.

Theorem 7**.**

Let $\rm{x}\in\{\rm{r},\rm{w}\}.$ Let $K_{\rm{low}},K_{\rm{up}},K,\alpha$ be as above, and let the assumptions of Theorem 3 hold. Then there exist constants $C_{0},C_{1}$ such that for all $d\in\mathbb{N}$ and all $L\geq d$ it holds

[TABLE]

and

[TABLE]

where $N=N(L,d)$ is the cardinality of information used by the algorithm $A(L,d).$

To prove Theorem 7 we need a lemma bounding $N(L,d)$ in terms of $K_{\rm{low}},K_{\rm{up}},K,d$ and $L.$

Lemma 8**.**

Let $K_{\rm{low}},K_{\rm{up}},K$ be as above. Put

[TABLE]

[TABLE]

[TABLE]

For every $d\in\mathbb{N}$ and $L\geq d$ it holds

[TABLE]

Moreover, if the building blocks of the Smolyak method use nested information then

[TABLE]

Proof.

We have

[TABLE]

Now following the steps of [64, Lemma 7] we obtain

[TABLE]

Now we provide a lower bound on $N(L,d).$ Note that given the cardinality of information used by the building blocks, the cardinality of information used by the Smolyak method is minimal when the information used by the building blocks is nested for every coordinate. In this case the information used by the Smolyak method is exactly the information used by $\sum_{|\operatorname{{\bf l}}|=L}\bigotimes_{n=1}^{d}U^{(n)}_{l_{n}}.$ Let us fix $\operatorname{{\bf l}}\in\mathbb{N}^{d},|\operatorname{{\bf l}}|=L.$ The expected value of the cardinality of information used by $\bigotimes_{n=1}^{d}U^{(n)}_{l_{n}}$ and at the same time not used by any other $\bigotimes_{n=1}^{d}U^{(n)}_{v_{n}}$ with $|{\bf v}|=L$ is

[TABLE]

We obtain

[TABLE]

The upper bound in the case when the building blocks use nested information follows in exactly the same manner on noting that

[TABLE]

∎

Proof.

(Theorem 7)

Note that $N(L,1)=m_{L,1}$ so we have already showed the statement for $d=1$ in (24). It remains to consider the case $d>1.$ Consider the function

[TABLE]

We will show that there exist constants $\widetilde{C}_{u,0},\widetilde{C}_{u,1},\widetilde{C}_{l,0},\widetilde{C}_{l,1}$ such that for $N_{u},N_{l}$ from Lemma 8 it holds

[TABLE]

and

[TABLE]

Now unimodality of $f$ combined with the fact that the extremum is a maximum yields $f(N(L,d))\geq\min\{f(N_{u}),f(N_{l})\}$ finishing the proof.

First we prove (29). Calling upon Theorem 3 and using $L\leq\frac{\log(N_{u})}{\log(K)}$ we get

[TABLE]

with constants $C_{u,0},C_{u,1}$ not depending neither on $d$ nor on $N(L,d).$ By Stirling’s formula we conclude

[TABLE]

Now we prove (30). To this end it suffices to prove that there exist constants $\hat{C}_{0},\hat{C}_{1}$ independent of $d$ and $N$ such that

[TABLE]

i.e.,

[TABLE]

Note that

[TABLE]

so, putting $\hat{K}=\frac{K}{K-1}\frac{K_{\rm{up}}}{K_{\rm{low}}}$ we have

[TABLE]

Since obviously $\left(\frac{N_{l}}{N_{u}}\right)^{\alpha}\leq 1$ this shows (31) and finishes the proof of the theorem. ∎

5.3 Lower Error Bounds

In this subsection we make the following additional assumptions.

The first assumption states that there exist a sequence of instances of the problem $\{S_{d},F_{d},G_{d},\mathcal{A}(\Omega,\Lambda)\}$ that is genuinely univariate, i.e., there exists a sequence $(f_{l})_{l\in\mathbb{N}}\in F_{d},\,f_{l}=g_{1,l}\otimes g_{2,l}\otimes\cdots\otimes g_{d,l}$ such that $\lVert f_{l}\rVert_{F_{d}}=1$ for which

[TABLE]

and the $U_{l}^{(n)},l\geq 1$ are exact on $g_{n,l}$ for $n>1$ .

Secondly, we assume that there exist constants $\widetilde{C}>0,\widetilde{D}\in(0,1)$ such that for every $l\in\mathbb{N}$

[TABLE]

Let us put

[TABLE]

with $\widetilde{D}$ as in (33) and $K$ as in (20). Using (33) and (21) one easily sees that

[TABLE]

meaning that we have $\beta\geq\mu^{(1)}_{{\rm{x}}},$ where $\mu^{(1)}_{{\rm{x}}}$ is as in (22). Moreover, by choosing $(g_{1,l})_{l\in\mathbb{N}}$ appropriately, $\beta$ can be made arbitrarily close to $\mu^{(1)}_{{\rm{x}}}.$

Example 9**.**

The assumptions made in this subsection are quite naturally met for many important problems. Consider for instance an integration problem as described in Example 1, where $F^{(n)},n=2,\ldots,d$ may be any spaces containing constant functions. Then, for an appropriate $(g_{1,l})_{l\in\mathbb{N}}$ (chosen so that the integration error does not converge too fast to [math]) we have that

[TABLE]

satisfies our assumptions for any randomized quadrature with weights adding up to $1.$

Lemma 10**.**

Let ${\rm{x}}\in\{\rm{w},\rm{r}\},$ and let (32) and (33) hold. Then there exists a constant $\hat{c}_{d}$ such that

[TABLE]

for all $L\geq d.$

If additionally (32) and (33) are satisfied for all $d\in\mathbb{N}$ with the same constants $\widetilde{C}$ and $\widetilde{D}$ and $\Theta:=\sup_{L,d}\left(\frac{m_{L,1}}{m_{L-d+1,1}}\right)^{2\beta}\theta^{2}_{d}$ is bounded, then we may choose the constants $(\hat{c}_{d})_{d\in\mathbb{N}}$ in such a way that they are all equal.

Proof.

Choosing $f_{l}$ satisfying (32), due to exactness assumption we obtain

[TABLE]

Let us put $T:=\bigotimes_{n=2}^{d}S^{(n)},h=\bigotimes_{n=2}^{d}g_{n,l}.$ Due to (34) we have

[TABLE]

∎

Remark 11**.**

In particular constants $(\hat{c}_{d})_{d\in\mathbb{N}}$ may be chosen all equal e.g. when (32) and (33) are satisfied for all $d\in\mathbb{N}$ with the same constants $\widetilde{C},\widetilde{D}$ and $\theta:=\sup_{d\in\mathbb{N}}\theta_{d}<\infty.$

Lemma 12**.**

Let there exist constants $1\leq K_{\rm{low}}\leq K_{\rm{up}},1<K$ such that for all $n=1,\ldots,d$ and $l\in\mathbb{N}$ (20) is satisfied. Then there exists a constant $\widetilde{c}_{d}$ such that

[TABLE]

for all $L\geq d.$ Moreover, if $\xi:=\xi(d):=(K^{d}-1)^{\frac{1}{d}}>1$ then there exists a constant $\widetilde{C}_{d}$ such that

[TABLE]

for all $L\geq d.$

Proof.

First we prove the upper bound.

On the one hand, due to (21) it holds

[TABLE]

where we used that the function $[0,\infty)\ni x\mapsto(K^{x}-1)^{\frac{1}{x}}$ is increasing.

On the other hand, according to Lemma 8

[TABLE]

It follows that the constant

[TABLE]

does the job.

Now we prove the lower bound.

On the one hand due to (21) we have

[TABLE]

Noticing that $f:[d,\infty)\rightarrow\mathbb{R},x\mapsto\frac{(x-1)\cdots(x-d+1)}{x^{d-1}}$ is increasing we obtain

[TABLE]

On the other hand we have due to Lemma 8

[TABLE]

It follows that the constant

[TABLE]

satisfies

[TABLE]

∎

Remark 13**.**

Note that the constants $\widetilde{C}_{d}$ and $\widetilde{c}_{d}$ from Lemma 12 fall superexponentially fast in $d.$

Corollary 14**.**

Let ${\rm{x}}\in\{\rm{r},\rm{w}\}.$ Let (32) and (33) hold. Furthermore, let there exist constants $1\leq K_{\rm{low}}\leq K_{\rm{up}},1<K$ such that for all $n=1,\ldots,d$ and $l\in\mathbb{N}$ (20) is satisfied. Moreover, assume that $m_{L,1}\geq 16$. Then there exists a constant $c_{d}$ such that given $\delta\in(0,1)$ there exists $N(\delta)$ such that for every $N\geq N(\delta)$

[TABLE]

with $\beta=\frac{\log(\frac{1}{\widetilde{D}})}{\log(K)}$ and $N=N(L,d).$

Proof.

Let $\widetilde{c}$ be such that for every $L\in\mathbb{N}$

[TABLE]

The existence of $\widetilde{c}$ is guaranteed by Lemma 12. We put $\widetilde{c}_{0}:=\min\{\widetilde{c},1\}.$

We would like to express the bound from Lemma 10 in terms of the cardinality $N:=N(L,d)$. To this end we want to find a function $g:\mathbb{R}\rightarrow\mathbb{R}$ of the form $g(x)=\frac{x^{\beta}}{(\log(x))^{\eta}}$ such that for large $m$

[TABLE]

implying

[TABLE]

We rewrite (35) as

[TABLE]

Hence (35) holds if

[TABLE]

and the expression on the right hand side converges from below to $(d-1)\beta$ as $m$ goes to $\infty.$ To obtain

[TABLE]

it is sufficient to check that g is increasing on the interval $[m_{L,1}\log(m_{L,1})^{d-1},\infty).$ Simple calculations reveal that $g$ is increasing on $[e^{\frac{\eta}{\beta}},\infty)\supset[e^{d-1},\infty).$ The final step is to notice that

[TABLE]

Putting $c_{d}:=\hat{c}_{d}\widetilde{c}_{0}^{\beta}$ finishes the proof. ∎

6 Application to Infinite-Dimensional Integration

In Theorem 18 we provide a sharp result on randomized infinite-dimensional integration on weighted reproducing kernel Hilbert spaces that parallels the sharp result on deterministic infinite-dimensional integration stated in [24, Theorem 5.1]. Results from [23] and from [52] in combination with Theorem 7 rigorously establish the sharp randomized result in the special case where the weighted reproducing kernel Hilbert space is based on an anchored univariate kernel. With the help of the embedding tools provided in [24] this result will be extended to general weighted reproducing kernel Hilbert spaces. Before we can state and prove Theorem 18 we first have to introduce the setting, cf. [24].

For basic results about reproducing kernels $K$ and the corresponding Hilbert spaces $H(K)$ we refer to [1]. We denote the norm on $H(K)$ by $\|\cdot\|_{K}$ and the space of constant functions (on a given domain) by $H(1)$; here $1$ denotes the constant kernel that only takes the function value one.

6.1 Assumptions

Henceforth we assume that

1. (A1)

$H$ is a vector space of real-valued functions on a domain $D\neq\emptyset$ with $H(1)\subsetneq H$

and

1. (A2)

$\|\cdot\|_{1}$ and $\|\cdot\|_{2}$ are seminorms on $H$, induced by symmetric bilinear forms $\langle\cdot,\cdot\rangle_{1}$ and $\langle\cdot,\cdot\rangle_{2}$, such that $\|1\|_{1}=1$ and $\|1\|_{2}=0$.

Let

[TABLE]

Furthermore, we assume that

1. (A3)

$\|\cdot\|_{H}$ is a norm on $H$ that turns this space into a reproducing kernel Hilbert space, and there exists a constant $c\geq 1$ such that

[TABLE]

Condition (37) is equivalent to the fact that $\|\cdot\|_{H}$ and $\left|\langle\cdot,1\rangle_{1}\right|+\|\cdot\|_{2}$ are equivalent norms on $H$.

Let us restate Lemma 2.1 from [24]:

Lemma 15**.**

For each $\gamma>0$ there exists a uniquely determined reproducing kernel $k_{\gamma}$ on $D\times D$ such that $H(1+k_{\gamma})=H$ as vector spaces and

[TABLE]

Moreover, the norms $\|\cdot\|_{H}$ and $\|\cdot\|_{1+k_{\gamma}}$ are equivalent and $H(1)\cap H(k_{\gamma})=\{0\}$.

Note that for the special value $\gamma=1$ we have $\|\cdot\|_{1+k_{1}}=\|\cdot\|_{H}$.

The next example illustrates the assumptions and the statement of Lemma 15; for more information and a slight generalization see [24, Example 2.3].

Example 16**.**

Let $D:=[0,1)$ and $r>1/2$. The periodic Sobolev space $K_{r}=K_{r}([0,1))$ (also known as Korobov space) is the Hilbert space of all $f\in L^{2}([0,1])$ with finite norm

[TABLE]

where $\hat{f}(h)=\int_{0}^{1}f(t)e^{-2\pi iht}\,\mathrm{d}t$ is the $h$-th Fourier coefficient of $f$. The functions in $K_{r}$ are continuous and periodic. It is easily checked that the reproducing kernel of $K_{r}$ is given by

[TABLE]

Consider the pair of seminorms on $K_{r}$ given by

[TABLE]

The assumptions (A1), (A2), and (A3) are easily verified. For $\gamma>0$ we have $k_{\gamma}=\gamma\cdot k_{1}$.

Further examples of spaces that satisfy the assumptions (A1), (A2), and (A3) are, for instance, the (non-periodic) Sobolev spaces $W^{r,2}([0,1])$ of smoothness $r\in\mathbb{N}$ endowed with either the standard norm, the anchored norm or the ANOVA norm, see [24, Example 2.1].

We now want to study weighted tensor product Hilbert spaces of multivariate functions, which implies that we have to consider product weights as introduced in [55]. More precisely, we consider a sequence ${\boldsymbol{\gamma}}=\left(\gamma_{j}\right)_{j\in\mathbb{N}}$ of positive weights that satisfies

[TABLE]

The decay of the weights is quantified by

[TABLE]

due to (39) we have $\operatorname{decay}({\boldsymbol{\gamma}})\geq 1$. For each weight $\gamma_{j}$ let $k_{\gamma_{j}}$ be the kernel from Lemma 15. With the help of the weights we can define spaces of functions of finitely many variables. For $d\in\mathbb{N}$ we define the reproducing kernel $K^{{\boldsymbol{\gamma}}}_{d}$ on $D^{d}\times D^{d}$ by

[TABLE]

The reproducing kernel Hilbert space $H(K^{{\boldsymbol{\gamma}}}_{d})$ is the (Hilbert space) tensor product of the spaces $H(1+k_{\gamma_{j}})$.

Now we want to define a space of functions of infinitely many variables. The natural domain for the counterpart of (40) for infinitely many variables is given by

[TABLE]

Let $a,a_{1},\dots,a_{n}\in D$ be arbitrary. Due to [24, Lemma 2.2] we have $(a_{1},\dots,a_{n},a,a,\dots)\in{\mathfrak{X}}^{{\boldsymbol{\gamma}}}$, and in particular ${\mathfrak{X}}^{{\boldsymbol{\gamma}}}\neq\emptyset$. We define the reproducing kernel $K^{{\boldsymbol{\gamma}}}_{\infty}$ on ${\mathfrak{X}}^{{\boldsymbol{\gamma}}}\times{\mathfrak{X}}^{{\boldsymbol{\gamma}}}$ by

[TABLE]

For a function $f\colon D^{d}\to\mathbb{R}$ we define $\psi_{d}f\colon{\mathfrak{X}}^{\boldsymbol{\gamma}}\to\mathbb{R}$ by

[TABLE]

Due to [24, Lemma 2.3] $\psi_{d}$ is a linear isometry from $H(K_{d}^{{\boldsymbol{\gamma}}})$ into $H(K^{{\boldsymbol{\gamma}}}_{\infty})$, and

[TABLE]

6.2 The Integration Problem

To obtain a well-defined integration problem we assume that $\rho$ is a probability measure on $D$ implying

[TABLE]

Let $\rho^{d}$ and $\rho^{\mathbb{N}}$ denote the corresponding product measures on $D^{d}$ and $D^{\mathbb{N}}$, respectively.

Due to [24, Lemma 3.1] we have for all $d\in\mathbb{N}$ that

[TABLE]

and the respective embeddings $J_{d}$ from $H(K_{d}^{\boldsymbol{\gamma}})$ into $L^{1}(D^{d},\rho^{d})$ are continuous with

[TABLE]

Define the linear functional $I_{d}\colon H(K_{d}^{\boldsymbol{\gamma}})\to\mathbb{R}$ by

[TABLE]

Note that $\|I_{d}\|_{\rm op}\geq 1$, since $I_{d}(1)=1$ and $\|1\|_{K^{\boldsymbol{\gamma}}_{d}}=1$. Furthermore, $\|I_{d}\|_{\rm op}\leq\|J_{d}\|_{\rm op}$, and therefore (45) implies

[TABLE]

This yields the existence of a uniquely determined bounded linear functional

[TABLE]

cf. [24, Lemma 3.2].

Note that every $f\in H(K^{\boldsymbol{\gamma}}_{\infty})$ is measurable with respect to the trace of the product $\sigma$-algebra on $D^{\mathbb{N}}$. (This follows from (44), (45), and the fact that the pointwise limit of measurable functions is again measurable.)

If ${\mathfrak{X}}^{\boldsymbol{\gamma}}$ is measurable, $\rho^{\mathbb{N}}({\mathfrak{X}}^{\boldsymbol{\gamma}})=1$, and $H(K^{\boldsymbol{\gamma}}_{\infty})\subseteq L^{1}({\mathfrak{X}}^{\boldsymbol{\gamma}},\rho^{\mathbb{N}})$, then the bounded linear functional (47) is given by

[TABLE]

For sufficient conditions under which these assumptions are fulfilled we refer to [27].

We consider the integration problem on $H(K^{\boldsymbol{\gamma}}_{\infty})$ consisting in the approximation of the functional $I_{\infty}$ by randomized algorithms that use function evaluations (i.e., standard information) as admissable information.

6.3 The Unrestricted Subspace Sampling Model

We use the cost model introduced in [40], which we refer to as unrestricted subspace sampling model. It only accounts for the cost of function evaluations. To define the cost of a function evaluation, we fix an anchor $a\in D$ and a non-decreasing function

[TABLE]

Put

[TABLE]

For each $u\in\mathcal{U}$ put

[TABLE]

To simplify the representation, we confine ourselves to non-adaptive randomized linear algorithms of the form

[TABLE]

where the number $n\in\mathbb{N}$ of knots is fixed and the knots ${\boldsymbol{t}}^{(i)}$ as well as the coefficients $w_{i}\in\mathbb{R}$ are random variables with values in some $\mathcal{T}_{v_{i}}$, $v_{i}\in\mathcal{U}$, and in $\mathbb{R}$, respectively. (We discuss a larger class of algorithms in Remark 21.) The cost of $Q$ is given by

[TABLE]

In the definition of the cost function an inclusion property has to hold for all $\omega\in\Omega$. Often this worst case point of view is replaced by an average case (cf., e.g., [42] or [9, 23, 52]). We stress that such a replacement would not affect the cost of the algorithms that we employ to establish our upper bounds for the $N$-th minimal errors; for lower bounds cf. Remark 21(iii).

Let $\rm{x}\in\{\rm{d},\rm{r},\rm{w}\}$. For $N\geq 0$ let us define the $N$-th minimal error on $H(K^{\boldsymbol{\gamma}}_{\infty})$ by

[TABLE]

where in the case $\rm{x}=\rm{d}$ the algorithms have to be deterministic, while in the case $\rm{x}\in\{\rm{r},\rm{w}\}$ they are allowed to be randomized. The (polynomial) convergence order of the $N$-th minimal errors of infinite-dimensional integration is given by

[TABLE]

In analogy to our definitions for infinite-dimensional integration, we consider for univariate integration on $H(1+k_{1})$ also linear randomized algorithms $Q$ of the form (48), except that this time the knots ${\boldsymbol{t}}^{(i)}$ are, of course, random variables with values in $D$. The cost of such an algorithm is simply the number $n$ of function evaluations, and $N$-th minimal errors on $H(1+k_{1})$ are given by

[TABLE]

The (polynomial) convergence order of the $N$-th minimal errors of univariate integration is given by

[TABLE]

Remark 17**.**

Let $K\in\{K^{{\boldsymbol{\gamma}}}_{\infty},1+k_{1}\}$ and, accordingly, $I\in\{I_{\infty},I_{1}\}$. Obviously,

[TABLE]

Furthermore, it is easy to see that $e^{\rm w}(N,K)\geq e^{\rm d}(N,K)$ holds: If $Q$ is an arbitrary randomized algorithm of the form (48) with $\operatorname{cost}(Q)\leq N$, then for every $\omega\in\Omega$ the cost of the deterministic algorithm $Q(\omega)$ is at most $N$, implying

[TABLE]

which in turn leads to $e^{\rm w}(I,Q)\geq e^{\rm d}(N,K)$. Hence we obtain

[TABLE]

6.4 A Sharp Result on Infinite-Dimensional Integration

The next theorem determines the exact polynomial convergence rate of the $N$-th minimal errors of infinte-dimensional integration on weighted reproducing kernel Hilbert spaces.

Theorem 18**.**

Let $\rm{x}\in\{\rm{r},\rm{w}\}$. If the cost function \$$ satisfies $(\nu)=\Omega(\nu)$and$$(\nu)=O(e^{\sigma\nu})$for some$\sigma\in(0,\infty)$, then we have

[TABLE]

Notice that the theorem implies that in the randomized setting infinite-dimensional integration on weighted reproducing kernel Hilbert spaces is (essentially) not harder than the corresponding univariate integration problem (as far as the polynomial convergence rate is concerned) as long as the weights decay fastly enough, i.e., as long as

[TABLE]

Proof.

Let us first consider the case ${\rm x}={\rm r}$. In the special case where the reproducing kernel $k_{1}$ is anchored in $a$ (i.e., $k_{1}(a,a)=0$) and satisfies $\gamma k_{1}=k_{\gamma}$ for all $\gamma>0$ (cf. Lemma 15), the statement of the Theorem follows from [23] and from [52] in combination with Theorem 7, as we will explain below in detail.

For a general reproducing kernel $k_{1}$ we need to find a suitably associated reproducing kernel $k_{a}$ anchored in $a$ and satisfying $\gamma k_{a}=(k_{a})_{\gamma}$ for all $\gamma>0$ to employ the embedding machinery from [24] to obtain the desired result (52). To this purpose we consider the bounded linear functional $\xi:H\to\mathbb{R}$, $f\mapsto f(a)$, where $a\in D$ is our fixed anchor. We define a new pair of seminorms on $H$ by

[TABLE]

Notice that $\|\cdot\|_{1,a}$ is induced by the symmetric bilinear form $\langle f,g\rangle_{1,a}:=\xi(f)\cdot\xi(g)$. This new pair of seminorms satisfies obviously assumption (A2) and the norms $\|\cdot\|_{H}=(\|\cdot\|^{2}_{1}+\|\cdot\|^{2}_{2})^{1/2}$ and $\|\cdot\|_{H,a}:=(\|\cdot\|^{2}_{1,a}+\|\cdot\|^{2}_{2,a})^{1/2}$ are equivalent norms on $H$. Hence $\|\cdot\|_{H,a}$ turns $H$ into a reproducing kernel Hilbert space, and satisfies (37) with $c=1$ since

[TABLE]

Thus the new pair of seminorms satisfies also (A3). Furthermore, if $k_{a}$ is the reproducing kernel on $D\times D$ such that

[TABLE]

and

[TABLE]

then $k_{a}$ is anchored in $a$ and moreover we have $H(1+k_{a})=H$ as vector spaces, $H(1)\cap H(k_{a})=\{0\}$, and

[TABLE]

for all $\gamma>0$, $f\in H$, implying $(k_{a})_{\gamma}=\gamma k_{a}$, see [24, Rem. 2.2]. Since $\|\cdot\|_{H}=\|\cdot\|_{1+k_{1}}$ and $\|\cdot\|_{H,a}=\|\cdot\|_{1+k_{a}}$ are equivalent norms on $H(1+k_{1})=H=H(1+k_{a})$, we obtain $\lambda^{{\rm r}}(1+k_{1})=\lambda^{{\rm r}}(1+k_{a})$. Due to [24, Thm. 2.3] we have

[TABLE]

According to (42) we define $K^{{\boldsymbol{\gamma}},a}_{\infty}:{\mathfrak{X}}^{{\boldsymbol{\gamma}}}\times{\mathfrak{X}}^{{\boldsymbol{\gamma}}}\to\mathbb{R}$ by

[TABLE]

Now we consider the integration problem in $H(K^{{\boldsymbol{\gamma}},a}_{\infty})$ and may use [23, Subsect. 3.2.1] and [52, Cor. 1] in combination with Theorem 7. Indeed, due to Theorem 7 we may choose linear randomized algorithms with convergence rates $\alpha$ arbitrarily close to $\lambda^{{\rm r}}(1+k_{1})=\lambda^{{\rm r}}(1+k_{a})$ to obtain via the randomized Smolyak method algorithms that satisfy (26) for ${\rm x}={\rm r}$ (and consequently also [52, Eqn. (10)]). Now [52, Cor. 1] ensures that

[TABLE]

Furthermore, we have due to [23, Eqn. (21)]

[TABLE]

Due to [24, Cor. 5.1] these estimates also hold for $H(K^{{\boldsymbol{\gamma}}}_{\infty})$.

Let us now consider the case ${\rm x}={\rm w}$. Due to (51), identity (52) follows directly from the deterministic result [24, Theorem 5.1]. ∎

We now provide two corollaries and to add some remarks.

Theorem 18, which deals with randomized algorithms, and the corresponding deterministic theorem [24, Theorem 5.1] allow immediately to compare the power of deterministic and randomized algorithms.

Corollary 19**.**

Let the assumptions of Theorem 18 hold. For infinite-dimensional integration on $H(K^{{\boldsymbol{\gamma}}}_{\infty})$ randomized algorithms are superior to deterministic algorithms, i.e., $\lambda^{{\rm r}}(K^{{\boldsymbol{\gamma}}}_{\infty})>\lambda^{{\rm d}}(K^{{\boldsymbol{\gamma}}}_{\infty})$, if and only if

[TABLE]

are satisfied.

The next corollary on infinite-dimensional integration on weighted Korobov spaces in the randomized setting parallels [24, Theorem 5.5], which discusses the deterministic setting.

Corollary 20**.**

Let $r>1/2$, and let the univariate reproducing kernel $k_{1}$ be as in (38). Then the weighted Korobov space $H(K^{\boldsymbol{\gamma}}_{\infty})$ is an infinite tensor product of the periodic Korobov space $H(1+k_{1})=K_{r}([0,1))$ of smoothness $r$, see Example 16. If the cost function \$$ satisfies $(\nu)=\Omega(\nu)$and$$(\nu)=O(e^{\sigma\nu})$for some$\sigma\in(0,\infty)$, then we have

[TABLE]

Proof.

Since $\lambda^{{\rm r}}(1+k_{1})=r+1/2$ and $\lambda^{{\rm w}}(1+k_{1})=r$ (see Appendix and Remark 17), Theorem 18 immediately yields the result for $\lambda^{{\rm r}}(K^{{\boldsymbol{\gamma}}}_{\infty})$ and $\lambda^{{\rm w}}(K^{{\boldsymbol{\gamma}}}_{\infty})$.

Notice that the result for $\lambda^{{\rm w}}(K^{{\boldsymbol{\gamma}}}_{\infty})$ can also be derived from Remark 17 and [24, Theorem 5.5]. ∎

Remark 21**.**

Let us come back to Theorem 18.

• (i)

Algorithms that achieve convergence rates arbitrarily close to $\lambda^{\rm x}(K^{{\boldsymbol{\gamma}}}_{\infty})$ are, e.g., multivariate decomposition methods (MDMs) that were introduced in **[40]** (in the deterministic setting) and developed further in **[52]** (in the deterministic and in the randomized setting); originally, these algorithms were called changing dimension algorithms, cf., e.g., **[8, 9, 23, 40, 52]**. MDMs exploit that the anchored function decomposition of an integrand can be efficiently computed; a method for multivariate integration based on the same idea is the dimension-wise integration method proposed in **[30]**. To achieve (nearly) optimal convergence rates, the MDMs may employ as building blocks Smolyak algorithms for multivariate integration that rely on (nearly) optimal algorithms for univariate integration on $H(1+k_{1})$, cf. **[52, Section 3.3]** and the proof of Theorem 18.

• (ii)

In the special case where ${\rm x}={\rm r}$ and where $k_{1}$ is an ANOVA-kernel (i.e., $k_{1}$ satisfies $\int_{D}k_{1}(y,x)\,{\rm d}x=0$ for every $y\in D$) a version of Theorem 18 was already proved in **[9, Theorem 4.3]**. It was the first result that rigorously showed that MDMs can achieve the optimal order of convergence also on spaces with norms that are not induced by an underlying anchored function space decomposition. It was not derived with the help of function space embeddings, but by an elaborate direct analysis. Apart from addressing only the ANOVA setting, a further drawback of **[9, Theorem 4.3]** is that its assumptions are slightly stronger than the ones made in Theorem 18: It is not sufficient to know the convergence rate of the $N$-th minimal errors of the univariate integration problem, but additionally one has to verify the existence of unbiased randomized algorithms for multivariate integration that satisfy certain variance bounds, see **[9, Assumption 4.1]**. Nevertheless, in many important cases it is well known that such variance bounds hold. Furthermore, one should mention that the analysis in **[9]** ist not restricted to product weights as in this section, but is done for general weights.

Note that the kernel $k_{1}$ of the Korobov space $K_{r}([0,1))$ from Example 16 and Corollary 20 is actually an ANOVA kernel. Hence the identity for $\lambda^{{\rm r}}(K^{{\boldsymbol{\gamma}}}_{\infty})$ in Corollary 20 may also be derived by employing **[9, Theorem 4.3]** after verifying the existence of unbiased algorithms for multivariate integration that satisfy **[9, Assumption 4.1]**.

• (iii)

The upper bound for $\lambda^{{\rm r}}(K^{{\boldsymbol{\gamma}}}_{\infty})$ in (52) relies on the corresponding bound **[23, Eqn. (21)]** for the case where the univariate reproducing kernel $k_{1}$ is anchored in $a$. Although the definition of the cost function in **[23]** takes the average case and not the worst case point of view and differs therefore from (49), both definitions lead to the same cost for the admissable class of algorithms $\mathcal{A}^{\rm res}$ considered in the unrestricted subspace sampling model in **[23]**. The class $\mathcal{A}^{\rm res}$ contains not only algorithms of the form (48), but also adaptive and non-linear algorithms. In the proof of Theorem 18 we employ the function space embeddings from **[24]**, which allows us to transfer results for linear algorithms from the case of anchored kernels to the general case. Hence we can conclude that the upper bound for $\lambda^{{\rm r}}(K^{{\boldsymbol{\gamma}}}_{\infty})$ in (52) holds also if we admit adaptive linear algorithms of the form (48) for infinite-dimensional as well as for univariate integration, but we do not know whether this is still the case if we admit non-linear algorithms.

We finish this section with some remarks on extensions of our results on infinte-dimensional integration to other settings.

Remark 22**.**

To obtain computational tractability of problems depending on a high or infinite number of variables, it is usually essential to be able to arrange the variables in such a way that their impact decays sufficiently fast. One approach to model the decreasing impact of successive variables is to use weighted function spaces, like the ones we defined and studied in this section, to moderate the influence of groups of variables. This approach goes back to the seminal paper [55]. Another approach is the concept of increasing smoothness with respect to properly ordered variables, see, e.g., [11, 25, 31, 37, 38, 49, 54]. The precise definition of Hilbert spaces of functions depending on infinitely many variables of increasing smoothness can be found in [25, Section 3]. Now [25, Theorem 3.19] shows how to relate these spaces to suitable weighted Hilbert spaces via mutual embeddings, making it therefore easy to transfer our results in the randomized setting, Theorem 18 and Corollary 20, from weighted spaces to spaces with increasing smoothness, cf. [25, Theorem 4.5 and Corollary 4.7] for the corresponding transference results in the deterministic setting.

Instead of applying our result Theorem 7 to the infinite-dimensional integration problem, we may also use it to tackle the infinite-dimensional $L_{2}$-approximation problem. Indeed, a sharp result for the latter problem was obtained in [63, Corollary 9] in the deterministic setting for weighted anchored reproducing kernel Hilbert spaces with the help of multivariate decomposition methods based on Smolyak algorithms (cf. [66, Theorem 7]). The analysis relies on explicit cost bounds for deterministic Smolyak algorithms from [64]. In [25, Theorem 4.5] the result is extended to weighted (not necessarily anchored) reproducing kernel Hilbert spaces (relying on the embedding tools from [24]) and to spaces of increasing smoothness.

Now one may use Theorem 7 to establish a corresponding result to [63, Corollary 9] for weighted anchored spaces in the randomized setting and may generalize it to non-anchored weighted spaces and to spaces of increasing smoothness via the embedding results established in [24, 25].

To work out all the details of these generalizations is beyond the scope of the present paper.

7 Appendix

7.1 Randomized Integration Error in Korobov Spaces

For $r>\frac{1}{2}$ we denote via $K_{r}:=K_{r}([0,1))$ the space of Korobov functions on a one-dimensional torus with smoothness parameter $r.$ The space is equipped with the norm

[TABLE]

see Example 16. It is a folklore result that the polynomial convergence order $\lambda^{\rm{r}}(1+k_{1})$ of randomized quadratures on $K_{r}$ is equal to $r+\frac{1}{2}.$ Since we have not found in the literature a complete proof handling all cases $r>\frac{1}{2}$, we decided to provide a proof sketch in this appendix. Similar reasoning for (non-periodic) Sobolev spaces with integer parameter $r$ may be found, e.g., in [44, Chapter  2.2]. For the lower error bound we need the following lemma.

Lemma 23**.**

Let $r>\frac{1}{2}.$ There exists a sequence of functions $(f_{n})_{n\in\mathbb{N}}\in K_{r}$ and a constant $C>0$ such that for every $n\in\mathbb{N}$

$\operatorname{{supp}}(f_{n})\subset(0,n^{-1})$, 2. 2.

$\int_{[0,1)}f_{n}\,dx=n^{-r-1},$** 3. 3.

$\lVert f_{n}\rVert^{2}_{r}\leq Cn^{-1}.$**

Proof.

Let $f$ be a positive infinitely many times differentiable function with $\operatorname{{supp}}(f)\subset(0,1)$ and integral equal to $1.$ Define

[TABLE]

The sequence $(f_{n})_{n\in\mathbb{N}}$ is the sequence we are looking for. Since all other properties are obvious it is enough to check that for $(f_{n})_{n},n\in\mathbb{N}$ the condition (3) holds. Fix $n.$ Since $f$ (as a bump function) is in the Schwartz space $\mathcal{S}(\mathbb{R}),$ the same holds for its Fourier transform $\hat{f}$ and as a result there exists $\tilde{C}>0$ such that for every $x\in\mathbb{R}$

[TABLE]

Since $|\hat{f_{n}}(0)|^{2}\leq n^{-3}$ we may neglect it. Simple calculations reveal

[TABLE]

Due to monotonicity of $\mathbb{N}\ni h\mapsto\frac{1}{n}\left(\frac{\tilde{C}}{(1+\frac{h}{n})^{r+1}}\right)^{2}\left(\frac{h}{n}\right)^{2r}$ we obtain

[TABLE]

which finishes the proof. ∎

For the upper bound we need a lemma on the $L^{2}-$approximation of functions from Korobov spaces by trigonometric polynomials.

Lemma 24**.**

Let $r>\frac{1}{2},f\in K_{r},N\in\mathbb{N}$, and let $q_{N}$ be the trigonometric polynomial defined by

[TABLE]

where

[TABLE]

are the $2N$ discrete Fourier coefficients of $f$. Then we have

[TABLE]

and

[TABLE]

The discrete Fourier coefficients $\widehat{\alpha}_{k}$, $k=1-N,2-N,\ldots,0,1,\ldots,N$, can be computed via the fast Fourier transform at cost $O(N\ln(N))$.

Proof.

Proofs of the statements of the lemma can be found in many standard texts on numerical analysis. We follow the course of [32, Sections 52 and 53]. Since $r>\frac{1}{2}$ we may write $f$ as a uniformly convergent Fourier series

[TABLE]

It is well-known (and not difficult to calculate) that if $q_{N}$ is a trigonometric polynomial of degree $N$ interpolating $f$ in the nodes $\frac{j}{2N},j=0,1,\ldots,2N-1$ then it is given by

[TABLE]

see for example Lemma $52.5$ in [32]. It holds

[TABLE]

The last sum may be bounded in an obvious way by $N^{-2r}\lVert f\rVert^{2}_{r}$ and for the double sum we may use the Cauchy Schwarz inequality to obtain

[TABLE]

The cost analysis of the fast Fourier transform is well known and can, e.g., be found in [32, Section 53]. ∎

Theorem 25**.**

Let $r>\frac{1}{2}.$ It holds

[TABLE]

Proof.

The upper bound on $\lambda^{\rm{r}}(1+k_{1})$ is settled immediately by Lemma 23 in conjunction with Corollary $7.35$ from [41]. Indeed, for $N\in\mathbb{N}$ choose $n:=6N$ and

[TABLE]

Then the assumptions of Corollary $7.35$ from [41] are satisfied for $\epsilon=n^{-r-1}$ implying $e^{\rm r}(N,1+k_{1})\geq\tfrac{1}{6^{r+1}\sqrt{2}}N^{-r-1/2}$, and thus establishing the upper bound.

To get the lower bound on $\lambda^{\rm{r}}(1+k_{1})$ consider the following algorithm. Let $N\in\mathbb{N}.$ We first interpolate $f\in K_{r}$ with the trigonometric polynomial $q_{N}$ from Lemma 24. Now the integral over $q_{N}$ is simply the discrete Fourier coefficient $\widehat{\alpha}_{0}$. To approximate the integral of $g:=(f-q_{N})$ we apply a simple Monte Carlo quadrature. To this end let $(X_{j})_{j=0}^{N-1}$ be independent random variables such that $X_{j}$ is distributed uniformly on $[0,1)$. We put

[TABLE]

Now, since $Qg$ is unbiased and $(X_{j})_{j=0}^{N-1}$ are independent, applying Lemma 24 we get

[TABLE]

Since cost of the algorithm is of order $O(N\ln(N))$, the claim follows. ∎

Acknowledgment

The authors thank Stefan Heinrich for pointing out the reference [34]. Part of the work was done while the authors were visiting the Mathematical Research and Conference Center Bedlewo in Autumn 2016 and the Erwin Schrödinger Institute for Mathematics and Physics (ESI) in Vienna in Autumn 2017. Both authors acknowledge support by the Polish Academy of Sciences; Marcin Wnuk additionally acknowledges support by the German Academic Exchange Service (DAAD).