Small Superposition Dimension and Active Set Construction for Multivariate Integration Under Modest Error Demand

TL;DR

This paper introduces an algorithm for constructing optimal active sets in multivariate integration, revealing that the superposition dimension can be surprisingly small under certain error and weight decay conditions.

Contribution

It proposes a new method for active set construction that produces significantly smaller sets and demonstrates that the superposition dimension is often very small for modest error demands.

Findings

Active sets can be much smaller with the new method.

Superposition dimension is at most 3 for error demands ≥ 10^{-3}.

Fast weight decay leads to smaller active sets.

Abstract

Constructing active sets is a key part of the Multivariate Decomposition Method. An algorithm for constructing optimal or quasi-optimal active sets is proposed in the paper. By numerical experiments, it is shown that the new method can provide sets that are significantly smaller than the sets constructed by the already existing method. The experiments also show that the superposition dimension could surprisingly be very small, at most 3, when the error demand is not smaller than $10^{-3}$ and the weights decay sufficiently fast.

Tables

Table 1. Table 1: | 𝔘 opt ​ ( 10 − 2 ) | superscript 𝔘 opt superscript 10 2 |{\mathfrak{U}}^{\mathrm{opt}}(10^{-2})| for p = 2 𝑝 2 p=2 and different a 𝑎 a , c 𝑐 c .

	$a$
$c$	4	3	2
$\frac{1}{2}$	3	5	12
1	4	7	30
2	6	14	122

Table 2. Table 3: | 𝔘 opt ​ ( 10 − 2 ) | superscript 𝔘 opt superscript 10 2 |{\mathfrak{U}}^{\mathrm{opt}}(10^{-2})| for p = ∞ 𝑝 p=\infty and different a 𝑎 a , c 𝑐 c .

	$a$
$c$	4	3	2
$\frac{1}{2}$	4	7	150
1	5	15	1346
2	8	43	31,013

Equations

$$f(\mathbf{x})=\sum _{\mathfrak{u}\subset {\mathbb{N}}_{+},|\mathfrak{u}|<\infty }{f}_{\mathfrak{u}}(\mathbf{x}).$$

$$\epsilon {\Vert \sum _{\mathfrak{u}\notin \mathfrak{U}(\epsilon )}{f}_{\mathfrak{u}}\Vert }_{{\mathcal{F}}_{\mathbf{\gamma }}}\text{mbox}forallf\in {\mathcal{F}}_{\gamma }.$$

$$\sum _{\mathfrak{u}\in \mathfrak{U}(\epsilon )}{f}_{\mathfrak{u}}.$$

$$d(\mathfrak{U}(\epsilon )):=\underset{\mathfrak{u}\in \mathfrak{U}(\epsilon )}{\max }|\mathfrak{u}|,$$

$$d\left({\mathfrak{U}}^{\mathrm{PW}}(\epsilon )\right)=O\left(\frac{\ln (1/\epsilon )}{\ln (\ln (1/\epsilon ))}\right)\text{mbox}as\epsilon \to 0.$$

$$d^{\sup }(\epsilon ):=\min \{d(\mathfrak{U}(\epsilon )):\mathfrak{U}(\epsilon )\text{mbox}isanactiveset\}.$$

$${\gamma }_{\mathfrak{u}}=\prod _{j\in \mathfrak{u}}{j}^{-a}\text{mbox}fora=2,3,4.$$

$$d^{\rm sup}(10^{-1})\,=\,\left\{\begin{array}[]{ll}1&\mbox{for\ }a=4,\\ 2&\mbox{for\ }a=3,\\ 2&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{and}\quad|{\mathfrak{U}}^{\rm PW}(10^{-1})|\,=\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 4&\mbox{for\ }a=3,\\ 6&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-2})\,=\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 2&\mbox{for\ }a=3,\\ 3&\mbox{for\ }a=2\end{array}\right.\quad\mbox{and}\quad|{\mathfrak{U}}^{\rm PW}(10^{-2})|\,=\,\left\{\begin{array}[]{ll}6&\mbox{for\ }a=4,\\ 8&\mbox{for\ }a=3,\\ 22&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-3})\,=\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 3&\mbox{for\ }a=3,\\ 4&\mbox{for\ }a=2\end{array}\right.\quad\mbox{and}\quad|{\mathfrak{U}}^{\rm PW}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}10&\mbox{for\ }a=4,\\ 22&\mbox{for\ }a=3,\\ 114&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-1})\,\leq\,\left\{\begin{array}[]{ll}1&\mbox{for\ }a=4,\\ 1&\mbox{for\ }a=3,\\ 2&\mbox{for\ }a=2,\end{array}\right.$$

$$|{\mathfrak{U}}^{\rm opt}(10^{-1})|\,=\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 2&\mbox{for\ }a=3,\\ 4&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{whereas}\quad|{\mathfrak{U}}^{\rm PW}(10^{-1})|\,=\,\left\{\begin{array}[]{ll}3&\mbox{for\ }a=4,\\ 5&\mbox{for\ }a=3,\\ 15&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-2})\,\leq\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 2&\mbox{for\ }a=3,\\ 3&\mbox{for\ }a=2\end{array}\right.$$

$$|{\mathfrak{U}}^{\rm opt}(10^{-2})|\,=\,\left\{\begin{array}[]{ll}4&\mbox{for\ }a=4,\\ 7&\mbox{for\ }a=3,\\ 30&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{whereas}\quad|{\mathfrak{U}}^{\rm PW}(10^{-2})|\,=\,\left\{\begin{array}[]{ll}8&\mbox{for\ }a=4,\\ 18&\mbox{for\ }a=3,\\ 158&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-3})\,\leq\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 3&\mbox{for\ }a=3,\\ 4&\mbox{for\ }a=2,\end{array}\right.$$

$$|{\mathfrak{U}}^{\rm opt}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}9&\mbox{for\ }a=4,\\ 24&\mbox{for\ }a=3,\\ 255&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{whereas}\quad|{\mathfrak{U}}^{\rm PW}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}20&\mbox{for\ }a=4,\\ 70&\mbox{for\ }a=3,\\ 1481&\mbox{for\ }a=2.\end{array}\right.$$

$$f_{\varnothing }+f_{\{1\}}(x_1)+\cdots +f_{\{5\}}(x_5)+f_{\{1,2\}}(x_1,x_2)+f_{\{1,3\}}(x_1,x_3)+f_{\{1,4\}}(x_1,x_4).$$

$$|{\mathfrak{U}}^{\mathrm{q}-\mathrm{opt}}(10^{-1})|=6\text{mbox}fora=2,|{\mathfrak{U}}^{\mathrm{q}-\mathrm{opt}}(10^{-2})|=32\text{mbox}fora=2,$$

$$\mbox{and}\quad|{\mathfrak{U}}^{\rm q-opt}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}26&\mbox{\ for\ }a=3,\\ 261&\mbox{\ for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-1})\,\leq\,\left\{\begin{array}[]{ll}1&\mbox{for\ }a=4,\\ 1&\mbox{for\ }a=3,\\ 3&\mbox{for\ }a=2,\end{array}\right.$$

$$|{\mathfrak{U}}^{\rm opt}(10^{-1})|\,=\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 3&\mbox{for\ }a=3,\\ 33&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{whereas}\quad|{\mathfrak{U}}^{\rm PW}(10^{-1})|\,=\,\left\{\begin{array}[]{ll}7&\mbox{for\ }a=4,\\ 21&\mbox{for\ }a=3,\\ 2358&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-2})\,\leq\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 2&\mbox{for\ }a=3,\\ 4&\mbox{for\ }a=2,\end{array}\right.$$

$$|{\mathfrak{U}}^{\rm opt}(10^{-2})|\,=\,\left\{\begin{array}[]{ll}5&\mbox{for\ }a=4,\\ 15&\mbox{for\ }a=3,\\ 1346&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{whereas}\quad|{\mathfrak{U}}^{\rm PW}(10^{-2})|\,=\,\left\{\begin{array}[]{ll}21&\mbox{for\ }a=4,\\ 149&\mbox{for\ }a=3\\ 120,935&\mbox{for\ }a=2.\end{array}\right.$$

$$d^{\rm sup}(10^{-3})\,\leq\,\left\{\begin{array}[]{ll}2&\mbox{for\ }a=4,\\ 3&\mbox{for\ }a=3,\\ 6&\mbox{for\ }a=2,\end{array}\right.$$

$$|{\mathfrak{U}}^{\rm opt}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}15&\mbox{for\ }a=4,\\ 83&\mbox{for\ }a=3,\\ 45,446&\mbox{for\ }a=2,\end{array}\right.\quad\mbox{whereas}\quad|{\mathfrak{U}}^{\rm PW}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}72&\mbox{for\ }a=4,\\ 923&\mbox{for\ }a=3.\end{array}\right.$$

$$|{\mathfrak{U}}^{\mathrm{q}-\mathrm{opt}}(10^{-1})|=38\text{mbox}fora=2,|{\mathfrak{U}}^{\mathrm{q}-\mathrm{opt}}(10^{-2})|=1904\text{mbox}fora=2,$$

$$\mbox{and}\quad|{\mathfrak{U}}^{\rm q-opt}(10^{-3})|\,=\,\left\{\begin{array}[]{ll}92&\mbox{\ for\ }a=3,\\ 52,159&\mbox{\ for\ }a=2.\end{array}\right.$$

$$\epsilon \Vert \mathcal{S}\Vert \Vert f\Vert \text{mbox}forallf\in {\mathcal{F}}_{\mathbf{\gamma }},$$

$$[{\boldsymbol{x}}_{\mathfrak{u}};{\boldsymbol{0}}_{{\mathfrak{u}}^{c}}]\,=\,[y_{1},y_{2},\dots]\quad\mbox{with}\quad y_{j}\,=\,\left\{\begin{array}[]{ll}x_{j}&\mbox{if\ }j\in{\mathfrak{u}},\\ 0&\mbox{if\ }j\notin{\mathfrak{u}}.\end{array}\right.$$

$$f^{(\mathfrak{u})}=\prod _{j\in \mathfrak{u}}\frac{\partial }{\partial x_j}f.$$

Full text

Small Superposition Dimension

and Active Set Construction

for Multivariate Integration

Under Modest Error Demand

A. D. Gilbert and G. W. Wasilkowski

**Abstract: ** Constructing active sets is a key part of the Multivariate Decomposition Method. An algorithm for constructing optimal or quasi-optimal active sets is proposed in the paper. By numerical experiments, it is shown that the new method can provide sets that are significantly smaller than the sets constructed by the already existing method. The experiments also show that the superposition dimension could surprisingly be very small, at most 3, when the error demand is not smaller than $10^{-3}$ and the weights decay sufficiently fast.

1 Introduction

In this short paper, we consider approximating integrals with infinitely many variables. We focus on approximations with a modest error demand, aiming at problems in, e.g., Mathematical Finance and Uncertainty Quantification, where the underlying stochastic process is not known and hence only rough approximations are needed. In our tests we use ${\varepsilon}=10^{-n}$ for $n=1,2,3$ as the error demands.

The functions to be integrated belong to $\{\gamma_{\mathfrak{u}}\}_{{\mathfrak{u}}\subset\mathbb{N}_{+}}$-weighted tensor product Banach spaces $\mathcal{F}_{\boldsymbol{\gamma}}$ which allow for the decomposition

[TABLE]

Here the summation is with respect to finite subsets ${\mathfrak{u}}$ of positive integers and each $f_{\mathfrak{u}}$ depends only on the variables $x_{j}$ with $j\in{\mathfrak{u}}$. We also assume that the weights $\gamma_{\mathfrak{u}}$ have a product form.

Integrals of such functions can be approximated by the Multivariate Decomposition Method, which is a refined version of the Changing Dimension Algorithm introduced in [4]. An essential part of those methods is the construction of an active set ${\mathfrak{U}}({\varepsilon})$ of subsets ${\mathfrak{u}}$ such that the integral of $\sum_{{\mathfrak{u}}\notin{\mathfrak{U}}({\varepsilon})}f_{\mathfrak{u}}$ can be neglected since it is bounded from above by

[TABLE]

In other words, it is enough to approximate integrals of the partial sum

[TABLE]

We would like to construct possibly small active sets and such that the largest cardinality among its elements ${\mathfrak{u}}$,

[TABLE]

is also small. This is because the partial sum $\sum_{{\mathfrak{u}}\in{\mathfrak{U}}({\varepsilon})}f_{\mathfrak{u}}$, that can be considered instead of the infinite sum $\sum_{{\mathfrak{u}}\subset\mathbb{N}_{+},|{\mathfrak{u}}|<\infty}$, has a small number $|{\mathfrak{U}}({\varepsilon})|$ of functions $f_{\mathfrak{u}}$, each depending on no more than $d({\mathfrak{U}}({\varepsilon}))$ variables.

A specific construction of such sets (denoted by ${\mathfrak{U}}^{\rm PW}({\varepsilon})$) was proposed in [7] and it was shown there that the largest cardinality among all ${\mathfrak{u}}\in{\mathfrak{U}}^{\rm PW}({\varepsilon})$ grows very slowly with decreasing ${\varepsilon}$,

[TABLE]

Moreover the size $|{\mathfrak{U}}^{\rm PW}({\varepsilon})|$ grows polynomially in $1/{\varepsilon}$. However, the asymptotic constants in the big-$O$ notation were not investigated and, as we shall see, they could be very large.

This is why in this paper we consider constructing possibly smallest active sets denoted by ${\mathfrak{U}}^{\rm opt}({\varepsilon})$. As we will show by examples, the difference between the size of ${\mathfrak{U}}^{\rm opt}({\varepsilon})$ and ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ could be very large. We also provide a construction of quasi-optimal sets, denoted by ${\mathfrak{U}}^{\rm q-opt}({\varepsilon})$, which sometimes are only slightly larger than the optimal ${\mathfrak{U}}^{\rm opt}({\varepsilon})$; however, their construction is less expensive.

We are also interested in active sets with the smallest $d({\mathfrak{U}}({\varepsilon}))$. This leads to the following concept of ${\varepsilon}$-superposition dimension (or superposition dimension for short) defined by

[TABLE]

Since the optimal active sets in our experiments have very small $d({\mathfrak{U}}^{\rm opt}({\varepsilon}))$, this implies that the superposition dimension is also small.

Note that our concept of the superposition dimension depends on the integration problem as well as the error demand ${\varepsilon}$. Hence it is in the same spirit as the definition of truncation dimension introduced recently in [3]. They are different from the definitions in statistical literature, see, e.g., [1, 5, 6, 9], where superposition and truncation dimensions are defined based on ANOVA decompositions and without any relation to the integration problem or the error demand ${\varepsilon}$. Moreover, the dimensions from [1, 5, 9] depend on specific functions, whereas the dimensions in [3] and in this paper are defined in the worst case sense, i.e., are relevant to all functions from the space $\mathcal{F}_{\boldsymbol{\gamma}}$.

Although the algorithms for constructing ${\mathfrak{U}}^{\rm q-opt}$ and ${\mathfrak{U}}^{\rm opt}$ work for rather general problems and spaces, we applied them to the integration problem and for weighted spaces of functions with mixed first order partial derivatives bounded in $L_{p}$ norms for $p\in[1,\infty]$. Such spaces have often been considered (mostly for $p=2$) when dealing with quasi-Monte Carlo methods.

The results depend on how fast the weights converge to zero. In the experiments, we considered

[TABLE]

For $p=1$, the construction from [7] is optimal, and it yields the following results:

[TABLE]

[TABLE]

[TABLE]

For $p>1$, ${\mathfrak{U}}^{\rm PW}$ are no longer optimal; however they are not much worse than optimal sets when $p$ is relatively close to 1. Moreover, for all the tests we have performed $d({\mathfrak{U}}^{\rm PW}({\varepsilon}))$ is very close to the superposition dimension. However the sizes (i.e., cardinalities) of ${\mathfrak{U}}^{\rm opt}({\varepsilon})$ and ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ could be very different, especially for $p=\infty$ and/or small $a$.

For instance, for $p=2$ we have the following results. In the case of ${\varepsilon}=10^{-1}$

[TABLE]

and

[TABLE]

For ${\varepsilon}=10^{-2}$

[TABLE]

and

[TABLE]

And finally, for ${\varepsilon}=10^{-3}$

[TABLE]

and

[TABLE]

The results for $a=4$ suggest that to achieve an error smaller than $10^{-3}$ it is enough to approximate $f({\boldsymbol{x}})=\sum_{{\mathfrak{u}}\subset\mathbb{N}_{+},|{\mathfrak{u}}|<\infty}f_{\mathfrak{u}}({\boldsymbol{x}})$ by

[TABLE]

As for the quasi-optimal sets, they are the same for $a=4$ and slightly larger for $a=3,2$:

[TABLE]

[TABLE]

For $p=\infty$ we have

[TABLE]

and

[TABLE]

Now for ${\varepsilon}=10^{-2}$

[TABLE]

and

[TABLE]

For ${\varepsilon}=10^{-3}$

[TABLE]

and

[TABLE]

For the tests above, the quasi-optimal active set was different from the corresponding optimal active set in the following cases only:

[TABLE]

[TABLE]

A collection of the active sets constructed above have been listed in full in the Appendix.

Our algorithms can also be used to construct the active sets where, instead of the standard worst case error, the normalized worst case error is used. More precisely, for the normalized worst case error we would like to have sets ${\mathfrak{U}}_{\rm norm}({\varepsilon})$ such that the integral of $\sum_{{\mathfrak{u}}\notin{\mathfrak{U}}_{\rm norm}({\varepsilon})}f_{\mathfrak{u}}$ is bounded by

[TABLE]

where $\|\mathcal{S}\|$ is the norm of the integration operator. Since in our case $\|\mathcal{S}\|\geq 1$, the corresponding active sets ${\mathfrak{U}}_{\rm norm}^{\rm X}({\varepsilon})$ are subsets of ${\mathfrak{U}}^{\rm X}({\varepsilon})$ for ${\rm X}\in\{{\rm opt,\,q-opt,\,PW}\}$ and could be even smaller.

2 Basic Definitions

We provide in this section basic concepts and definitions for special spaces of functions that are very often assumed in the literature, especially in the context of quasi-Monte Carlo methods. The presented algorithms can easily be modified to more general spaces.

2.1 ${\boldsymbol{\gamma}}$-Weighted Spaces

We follow here [2]. For $D=[0,1]$, let $\mathcal{D}=D^{\mathbb{N}_{+}}$ be the set of sequences (points) ${\boldsymbol{x}}=[x_{1},x_{2},\dots]$ with $x_{i}\in D$. Here $\mathbb{N}_{+}$ is the set of positive integers and we will use ${\mathfrak{u}}$ and ${\mathfrak{v}}$ to denote finite subsets of $\mathbb{N}_{+}$. We will also use the following notation: For ${\boldsymbol{x}}\in\mathcal{D}$ and ${\mathfrak{u}}$, by $[{\boldsymbol{x}}_{\mathfrak{u}};{\boldsymbol{0}}_{{\mathfrak{u}}^{c}}]$ we denote the point in $\mathcal{D}$ such that

[TABLE]

Next,

[TABLE]

For given

[TABLE]

let $\mathcal{F}_{{\boldsymbol{\gamma}},p}$ be the Banach space of functions defined on $\mathcal{D}$ with the following norm

[TABLE]

Of course, for $p=\infty$,

[TABLE]

We assume that the numbers $\gamma_{\mathfrak{u}}$ (called weights) are of product form (see [8])

[TABLE]

In general choosing the weights (in our case choosing $a$ and $c$) for a specific integral or application is a difficult problem which we do not attempt to address here. We assume that the parameters $a$ and $c$ are given with the problem.

It was shown in [2] that any $f\in\mathcal{F}_{{\boldsymbol{\gamma}},p}$ admits a unique decomposition, called the anchored decomposition,

[TABLE]

with $f_{\mathfrak{u}}$ given by

[TABLE]

where $(x_{j}-t_{j})^{0}_{+}$ is $1$ if $x_{j}>t_{j}$ and [math] otherwise. The functions $f_{\mathfrak{u}}$ belong to the following Banach spaces $F_{\mathfrak{u}}$

[TABLE]

Of course, $F_{\emptyset}$ is the space of constant functions with the absolute value as its norm. The spaces $F_{\mathfrak{u}}$ for ${\mathfrak{u}}\not=\emptyset$ are anchored at [math] since $f_{\mathfrak{u}}({\boldsymbol{x}})=0$ if there is $j\in{\mathfrak{u}}$ with $x_{j}=0$. This is why

[TABLE]

The space $\mathcal{F}_{{\boldsymbol{\gamma}},p}$ contains in particular the following class of functions.

Example 1

For a smooth function $g:\mathbb{R}\to\mathbb{R}$ and fast decaying numbers $a_{1},a_{2},\dots$, consider

[TABLE]

Then

[TABLE]

Hence $f\in\mathcal{F}_{{\boldsymbol{\gamma}},p}$ if the derivatives of $g$ and the coefficients $a_{j}$ satisfy

[TABLE]

2.2 Integration Problem

Consider the following integration functional

[TABLE]

given by

[TABLE]

Let $p^{*}$ denote the conjugate of $p$,

[TABLE]

We assume that

[TABLE]

since the left hand side of (3) is the norm of $\mathcal{S}$, i.e., (3) is a necessary and sufficient condition for continuity of $\mathcal{S}$. Indeed, letting $S_{\mathfrak{u}}$ be the restriction of $\mathcal{S}$ to $F_{\mathfrak{u}}$, we have that

[TABLE]

which, with an application of Hölder’s inequality, yields (3). For product weights of the form (1), we have

[TABLE]

Hence for product weights, (3) is equivalent to $a\,>\,1/{p^{*}}$. For the remainder of the paper it is assumed that $a\,>\,1/p^{*}$.

A very important part of the Multivariate Decomposition Method (MDM for short) is a construction of active sets ${\mathfrak{U}}({\varepsilon})$, i.e., sets that satisfy

[TABLE]

The essence of (4) is that, when approximating $\mathcal{S}(f)$, it is enough to restrict the attention to functions

[TABLE]

since any algorithm approximating $\sum_{{\mathfrak{u}}\in{\mathfrak{U}}({\varepsilon})}S_{\mathfrak{u}}(f_{\mathfrak{u}})$ with the worst case error on $\bigoplus_{{\mathfrak{u}}\in{\mathfrak{U}}({\varepsilon})}F_{\mathfrak{u}}$ bounded by ${\varepsilon}$ has its worst case error on the whole space $\mathcal{F}_{{\boldsymbol{\gamma}},p}$ bounded by

[TABLE]

The factor of $2^{1/p^{*}}$ is the result of applying Hölder’s inequality, see, e.g., [3]. Clearly, there are many sets satisfying (4), and we would like to construct possibly small active sets.

Definition 2

We say that an active set, denoted by ${\mathfrak{U}}^{\rm opt}({\varepsilon})$ is optimal, if

[TABLE]

We also define the ${\varepsilon}$-superposition dimension as the smallest $d({\mathfrak{U}}({\varepsilon}))$ among all active sets,

[TABLE]

3 Constructing Active Sets ${\mathfrak{U}}({\varepsilon})$

A construction of active sets was first proposed in [7]. The corresponding sets will be denoted by ${\mathfrak{U}}^{\rm PW}({\varepsilon})$. It was shown there that

[TABLE]

and that the cardinality of ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ is polynomial in $1/{\varepsilon}$. It is easy to see that for $p=1$, ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ are optimal. However, as we shall see, their size might be too large for large values of $p$, especially for $p=\infty$.

Let ${\mathfrak{U}}$ be a set of subsets ${\mathfrak{u}}$. Then

[TABLE]

Hence we are looking for ${\mathfrak{U}}({\varepsilon})$ such that

[TABLE]

Since Hölder’s inequality is sharp, (5) is equivalent to (4).

For the sake of completeness, we recall the construction for $p=1$, see [7].

3.1 Case $p=1$

For $p=1$, we have $p^{*}=\infty$ and $\|S_{\mathfrak{u}}\|_{F_{\mathfrak{u}}}=1$ for all ${\mathfrak{u}}$. Hence

[TABLE]

which for product weights reduces to $\sup_{{\mathfrak{u}}\notin{\mathfrak{U}}({\varepsilon})}\prod_{j\in{\mathfrak{u}}}c\,j^{-a}$. Therefore

[TABLE]

It is easy to see that ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ is the smallest set satisfying (4), i.e., it is a subset of any ${\mathfrak{U}}({\varepsilon})$ satisfying (5).

The examples of ${\mathfrak{U}}^{\rm PW}$ for specific values of $a$ and ${\varepsilon}$ are presented in the Appendix. For simplicity we use $c=1$ there.

3.2 Case of $p>1$

For $p>1$, the conjugate $p^{*}$ is finite and the construction of ${\mathfrak{U}}({\varepsilon})$ is more complicated.

We begin by recalling the construction of ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ in [7]. To simplify the notation, let

[TABLE]

For given ${\varepsilon}$ and $p$, a special threshold is computed and all ${\mathfrak{u}}$ with $\overline{\gamma}_{\mathfrak{u}}$ exceeding the threshold are included in the active set. More precisely, for $t\in(1/(ap^{*}),1)$ a threshold is given by

[TABLE]

Note that the interval $(1/(ap^{*}),1)$ is non-empty by the assumption that $a\,>\,1/p^{*}$ introduced in Section 2.2. In our numerical experiments we approximated the sum of $\overline{\gamma}_{\mathfrak{u}}^{\,t}$ for $t=i/40$ ($39\geq i>40/(ap^{*})$) and selected the value which resulted in the largest ${\rm Threshold}({\varepsilon},t)$. The approximations are calculated in a similar way as the computation of $A_{s}$ explained later (see (7)).

Clearly, ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ contains a number of ${\mathfrak{u}}$’s with the largest $\overline{\gamma}_{\mathfrak{u}}$; however, the number of them could be much larger than needed.

The collection of those ${\mathfrak{u}}$ with the largest $\overline{\gamma}_{\mathfrak{u}}$ that are necessary for (4) would result in the optimal set ${\mathfrak{U}}^{\rm opt}({\varepsilon})$. Since this optimal set is always a subset of ${\mathfrak{U}}^{\rm PW}({\varepsilon})$, the good property

[TABLE]

is preserved. More precisely, let $({\mathfrak{u}}_{j})_{j\in\mathbb{N}_{+}}$ be a sequence of all subsets ${\mathfrak{u}}$ ordered so that

[TABLE]

Then

[TABLE]

with $k=k({\varepsilon})$ such that

[TABLE]

The problem with this approach is that we do not know a priori the number $k=k({\varepsilon})$ and ordering a large number of $\overline{\gamma}_{\mathfrak{u}}$ might be too expensive. This is why the numbers $\overline{\gamma}_{\mathfrak{u}}$ will be ordered on-line. Actually, we propose two ways of constructing active sets. The first and simpler one produces what we call, quasi-optimal sets ${\mathfrak{U}}^{\rm q-opt}({\varepsilon})$ and it uses a partial ordering of $\overline{\gamma}_{\mathfrak{u}}$. The second one, uses ordering of $\overline{\gamma}_{\mathfrak{u}}$ and produces optimal sets ${\mathfrak{U}}^{\rm opt}({\varepsilon})$. However, as we will see the difference between both sets is very small; sometimes these sets are equal.

The numbers $\overline{\gamma}_{\mathfrak{u}}$ have the following properties that are crucial for our construction of quasi-optimal and optimal sets. Let $\ell$ be a given cardinality. In what follows we will write

[TABLE]

The first property is: If

[TABLE]

then

[TABLE]

The other property is: For $\ell+1\geq c^{1/a}$,

[TABLE]

We are ready to describe the constructions of active sets. First we need to approximate

[TABLE]

from above and with the relative error significantly smaller than ${\varepsilon}^{p^{*}}$. This can be done as follows. For a large natural number $s$

[TABLE]

It is easy to see that the relative error between $A$ and its approximation $A_{s}$ is proportional to $1/s^{2a\,p^{*}-2}$ with the asymptotic constant $c^{p^{*}}/((p^{*}+1)\,2^{ap^{*}-1})\,\prod_{j=1}^{\infty}(1+(c/j^{a})^{p^{*}}/(p^{*}+1))$.

A general idea of our construction is to select sets ${\mathfrak{u}}$ with large $\overline{\gamma}_{\mathfrak{u}}$ and subtract $\overline{\gamma}_{\mathfrak{u}}$ from $A_{s}$. This is repeated until $A_{s}$ is reduced to or below ${\varepsilon}^{p^{*}}$.

More specifically, consider a partition of $\mathbb{R}_{+}$ into intervals $I_{i}$ such that the numbers in $I_{j}$ are greater than those in $I_{j+1}$. For simplicity, we used

[TABLE]

in our numerical experiments when constructing quasi-optimal sets. However, we think that a better partition is possible, especially when constructing optimal active sets. We also associate with every interval a list $L_{j}$ that contains those ${\mathfrak{u}}$ for which $\overline{\gamma}_{\mathfrak{u}}$ has been subtracted from $A_{s}$ in $j$th step.

In the first $j=1$ step, add the empty set to $L_{1}$ and subtract $\overline{\gamma}_{\emptyset}=1$ from $A_{s}$. If the new $A_{s}$ satisfies $A_{s}\leq{\varepsilon}^{p^{*}}$, then terminate. Otherwise consider non-empty sets ${\mathfrak{u}}$ in the order of increasing cardinalities. Hence start with singleton sets ${\mathfrak{u}}=\{i\}$ for $i=1,\dots,k$, where $k$ is the largest integer such that $\overline{\gamma}_{\{k\}}$ is in $I_{1}$. Place $\{k+1\}$ into list $L_{2}$, and start subtracting from $A_{s}$ the values $\overline{\gamma}_{\{i\}}$ and store $\{i\}$ in $L_{1}$ until either the difference becomes less than or equal to ${\varepsilon}^{p^{*}}$, in which case we terminate, or $i=k$. Next repeat the same for sets of cardinality $2$, starting with sets $\{1,i\}$ for $i\leq k$, where now $k$ is the largest integer such that $\overline{\gamma}_{\{1,k\}}\in I_{1}$. Store $\{1,k+1\}$ in $L_{2}$. Next consider sets $\{2,i\}$, $\{3,i\}$, etc., until either all cardinality 2 sets corresponding to the current interval have been visited or the new value of $A_{s}$ is $\leq{\varepsilon}^{p^{*}}$, in which case we terminate. Continue working through the sets in order of increasing cardinality $\ell$ until $\overline{\gamma}_{\{1,2,\dots,\ell\}}\notin I_{1}$ and $\ell\geq c$ (of course, $\ell$ is always at least $c$ for $c\leq 1$). Then move to step $j=2$. The procedure in this (and later) steps is very similar except that for a fixed cardinality of ${\mathfrak{u}}$, we check if any such set has already been placed in $L_{2}$ in the 1st step. If it has, we start working with such sets first. For instance, for cardinality $1$, if $\{k+1\}\in L_{2}$, then we begin with sets $\{i\}$ for $i\geq k+1$. For cardinality $2$, if $\{i_{i},i_{2}\}$ has been placed in $L_{2}$, then we inspect sets $\{i_{1},i\}$ for $i\geq i_{2}$, before any other sets of cardinality two are considered. Once we find $\overline{\gamma}_{\{i,i+1\}}\notin I_{2}$, we store $\{i,i+1\}$ in $L_{3}$ and proceed to sets of cardinality 3, etc.

At the very end, ${\mathfrak{U}}^{\rm q-opt}({\varepsilon})$ consists of all subsets ${\mathfrak{u}}$ whose values $\overline{\gamma}_{\mathfrak{u}}$ were subtracted from $A_{s}$. The main procedure is outlined in Algorithm 2. In all of the algorithms $j_{\max}$ and $\ell_{\max}$ are computational thresholds denoting, respectively, the maximum number of intervals to be searched through and the maximum allowed cardinality of sets.

To search through the sets in a systematic way, we must keep track of the current set, ${\mathfrak{u}}$, and the index, $i$, that we are incrementing from. The subroutine increment-${\mathfrak{u}}$ outlined below in Algorithm 1 details how to increment ${\mathfrak{u}}$ from index $i$.

To make the presentation clearer Algorithm 2 is broken into two parts: First, we search starting from the sets found in the previous interval, which is handled by the subroutine q-opt-search in Algorithm 3. Then we continue searching through sets in order of increasing cardinality (line 9) starting where q-opt-search finished, at cardinality $\ell_{\mathrm{next}}$. The basic search structure is the same, however in q-opt-search each set we visit is checked to reduce multiple visits to a single set and ensure that the same set is not added to ${\mathfrak{U}}^{\mathrm{q-opt}}({\varepsilon})$ more than once.

The notation

[TABLE]

denotes that we call q-opt-search with inputs ${\mathfrak{U}}^{\mathrm{q-opt}}({\varepsilon})$, $T$, $(\bar{\gamma}_{\mathfrak{u}})_{|{\mathfrak{u}}|<\infty}$, $L_{j}$, $L_{j+1}$, $I_{j}$ and then use the output to update ${\mathfrak{U}}^{\mathrm{q-opt}}({\varepsilon})$, $T$, $\ell_{\mathrm{next}}$ and $L_{j+1}$.

The construction of optimal active sets is very similar. The main difference is that in the $j$th step, we first create the list $L_{j}^{\rm unsorted}$, order its elements ${\mathfrak{u}}\in L_{j}^{\rm unsorted}$ according to decreasing values of $\overline{\gamma}_{{\mathfrak{u}}}$, and next start subtracting the values $\overline{\gamma}_{\mathfrak{u}}$ from $A_{s}$. Again the lists $L_{j}$ will hold the sets visited in the previous interval.

In fact, if we do not care whether or not all of the sets are ordered but only that ${\mathfrak{U}}^{\rm opt}({\varepsilon})$ consists of the sets with the largest weights, then we only need to sort the sets which come from the final interval. This is because at the previous intervals all of the sets will need to be added to ${\mathfrak{U}}^{\rm opt}({\varepsilon})$, regardless of sorting. To do this in practice, for each interval we store the sum of all the weights corresponding to that interval. In Algorithm 4 we denote this by $T_{j}$. At the end of the $j$th step, we check whether $I_{j}$ is the final interval, i.e., if $A_{s}-\sum_{i=1}^{j}T_{i}\leq\varepsilon^{p^{\star}}$, if so we sort the sets and add them one-by-one until the active set is complete. Otherwise we add all of the sets in $L_{j}^{\rm unsorted}$ to ${\mathfrak{U}}^{\rm opt}({\varepsilon})$ and go to the next interval. For completeness, the construction of optimal active sets is detailed separately below in Algorithm 4 and the subroutine opt-search in Algorithm 5.

4 Discussion

In this paper we have introduced the notion of superposition dimension and optimal active sets to be used in the MDM for multivariate integration and presented an algorithm detailing their construction. We also introduced a second simplified, computationally less intensive version of the algorithm, which constructs quasi-optimal active sets. Our numerical results show that the quasi-optimal active sets are of a similar size to the optimal active sets. Often the two sets are exactly the same. In all of our numerical results the optimal and quasi-optimal active sets are smaller than, and have superposition dimension less than or equal to, the active sets using the construction in [7].

To observe how different choices of parameters $a$ and $c$ affect our construction, statistics on the resulting optimal active sets are given in Tables 2-4. Tables 2 and 2 give, respectively, the size and the superposition dimension of the optimal active set for $p=2$ and an error request of $10^{-2}$. For $p=\infty$, ${\varepsilon}=10^{-2}$ the size and superposition dimension of the optimal active sets are given in Tables 4 and 4. The results for the quasi-optimal active set are again very similar and so have not been included here. As expected these results demonstrate that as the decay of the weights is slower or the weights become larger ($a$ smaller and $c$ larger) the problem becomes more difficult and the active sets are by necessity larger. However the superposition dimension remains relatively small, at most 6.

Finally, we have constructed the sets ${\mathfrak{U}}^{\rm q-opt}({\varepsilon})$, ${\mathfrak{U}}^{\rm opt}({\varepsilon})$, and ${\mathfrak{U}}^{\rm PW}({\varepsilon})$ for the product weights with $c=1$. They are listed explicitly in the Appendix.

Acknowledgements

The authors would like to thank the two anonymous referees for their constructive comments which helped improve the paper.

Appendix

We list here a selection of the constructed active sets from the previous sections (the very largest sets have been omitted). To save the space sometimes we write $[...\{x_{1},\dots,x_{k},x_{k+1}\}]$ to denote the sequence of sets,

[TABLE]

For instance $[...\{3\}]$ denotes $\{1\},\{2\},\{3\}$ and $[...\{1,5\}]$ denotes $\{1,2\},\{1,3\},\{1,4\},\{1,5\}$.

Case $p=1$ and $a=4$

[TABLE]

Case of $p=1$ and $a=3$

[TABLE]

[TABLE]

Case $p=1$ and $a=2$

[TABLE]

[TABLE]

[TABLE]

Case $p=2$ and $a=4$

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Case of $p=2$ and $a=3$

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Case of $p=2$ and $a=2$

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Case $p=\infty$ and $a=4$

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Case $p=\infty$ and $a=3$

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Case $p=\infty$ and $a=2$

[TABLE]

[TABLE]

[TABLE]