Optimal experimental design that minimizes the width of simultaneous confidence bands

TL;DR

This paper introduces a novel optimal experimental design method that minimizes the width of simultaneous confidence bands in curvilinear regression models by leveraging geometric and group invariance principles.

Contribution

It proposes the tube-volume optimal (TV-optimal) design based on the volume of a tube, applicable to Fourier and weighted polynomial regressions, and explores its invariance properties.

Findings

TV-optimal design minimizes confidence band width.

The design forms an orbit of the M"obius group.

For specific regressions, TV-optimal designs include D-optimal designs.

Abstract

We propose an optimal experimental design for a curvilinear regression model that minimizes the band-width of simultaneous confidence bands. Simultaneous confidence bands for curvilinear regression are constructed by evaluating the volume of a tube about a curve that is defined as a trajectory of a regression basis vector (Naiman, 1986). The proposed criterion is constructed based on the volume of a tube, and the corresponding optimal design that minimizes the volume of tube is referred to as the tube-volume optimal (TV-optimal) design. For Fourier and weighted polynomial regressions, the problem is formalized as one of minimization over the cone of Hankel positive definite matrices, and the criterion to minimize is expressed as an elliptic integral. We show that the M\"obius group keeps our problem invariant, and hence, minimization can be conducted over cross-sections of orbits. We demonstrate that for the weighted polynomial regression and the Fourier regression with three bases, the tube-volume optimal design forms an orbit of the M\"obius group containing D-optimal designs as representative elements.

Tables

Table 1. Table 1: Upper α 𝛼 \alpha quantiles w α subscript 𝑤 𝛼 w_{\alpha} for the designs 𝒟 ​ ( v ) 𝒟 𝑣 \mathcal{D}(v) .

$v$	${\mathrm{Vol}}_1\left({\gamma }_{\Sigma }\right)$	$w_{0.1}$	$w_{0.05}$
$1/12$	10.872	2.3473  (2.3879)	2.6328  (2.6624)
$1/9$	10.697	2.3463  (2.3810)	2.6319  (2.6562)
$1/6$	10.469	2.3438  (2.3720)	2.6283  (2.6481)
$1/4$	10.304	2.3412  (2.3653)	2.6248  (2.6421)
$1/3$	10.260	2.3398  (2.3635)	2.6234  (2.6405)
$1/2$	10.383	2.3411  (2.3685)	2.6251  (2.6450)

Equations

$$y_{i}=b^{\top}f(x_{i})+\varepsilon_{i},\quad\varepsilon_{i}\sim N\bigl{(}0,\sigma^{2}(x)\bigr{)}\ \ \mbox{i.i.d.},$$

$$\Sigma =M^{-1}\text{mbox}withM=\sum _{i=1}^{N}f(x_i)f(x_i{)}^{\top }\frac{1}{{\sigma }^2(x_i)},$$

$${\left\{\begin{array}{c}{x}_i\\ p_i\end{array}\right\}}_{1\le i\le N}=\left\{\begin{array}{ccc}{x}_1& \cdots & x_N\\ \frac{1}{N}& \cdots & \frac{1}{N}\end{array}\right\}.$$

$$b^{\top }f(x)\in {\hat{b}}^{\top }f(x)\pm c_{\alpha }\sqrt{f(x{)}^{\top }\Sigma f(x)},$$

$$\displaystyle\Pr\biggl{(}\,\frac{|\,\widehat{b}^{\top}f(x)-b^{\top}f(x)|}{\sqrt{f(x)^{\top}\Sigma f(x)}}<c,\,\forall x\in\mathcal{X}\biggr{)}$$

$$=1-\alpha ,$$

$$\psi_{\Sigma}(x)=\frac{\Sigma^{\frac{1}{2}}f(x)}{\|\Sigma^{\frac{1}{2}}f(x)\|},\qquad\gamma_{\Sigma}=\bigl{\{}\pm\psi_{\Sigma}(x)\mid x\in\mathcal{X}\bigr{\}},$$

$${\gamma }_{\Sigma }\subset {\mathbb{S}}^{n-1}=\{u\in {\mathbb{R}}^n\mid \Vert u\Vert =1\}.$$

$$\Pr\biggl{(}\max_{x\in\mathcal{X}}\frac{|\,(\widehat{b}-b)^{\top}f(x)|}{\|\Sigma^{\frac{1}{2}}f(x)\|}>c\biggr{)}\sim\frac{\mathrm{Vol}_{1}(\gamma_{\Sigma})}{2\pi}\Pr\bigl{(}\chi^{2}_{2}>c^{2}\bigr{)}.$$

$$\frac{\mathrm{Vol}_{1}(\gamma_{\Sigma})}{2\pi}\Pr\bigl{(}\chi^{2}_{2}>c^{2}\bigr{)}+\chi(\gamma_{\Sigma})\Pr\bigl{(}\chi^{2}_{1}>c^{2}\bigr{)},$$

$$\frac{\mathrm{Vol}_{1}(\gamma_{\Sigma})}{2\pi}\Pr\bigl{(}\chi^{2}_{2}>c_{\alpha}^{2}\bigr{)}=\alpha.$$

$$\Pr\biggl{(}\max_{x\in\mathcal{X}}\frac{|\,(\widehat{b}-b)^{\top}f(x)|}{\|\Sigma^{\frac{1}{2}}f(x)\|}>c\biggr{)}\leq\frac{\mathrm{Vol}_{1}(\gamma_{\Sigma})}{2\pi}\Pr\bigl{(}R_{2}^{2}>c^{2}\bigr{)}+\chi(\gamma_{\Sigma})\Pr\bigl{(}R_{1}^{2}>c^{2}\bigr{)},$$

$${\mathrm{Vol}}_1({\gamma }_{\Sigma })$$

$$\displaystyle=2\int_{\mathcal{X}}\biggl{\|}\frac{\mathrm{d}}{\mathrm{d}x}\biggl{(}\frac{\Sigma^{\frac{1}{2}}f(x)}{\|\Sigma^{\frac{1}{2}}f(x)\|}\biggr{)}\biggr{\|}\,\mathrm{d}x$$

$$=2{\int }_{\mathcal{X}}\frac{\sqrt{(f(x{)}^{\top }\Sigma f(x))(g(x{)}^{\top }\Sigma g(x))-(f(x{)}^{\top }\Sigma g(x){)}^2}}{f(x{)}^{\top }\Sigma f(x)}\mathrm{d}x$$

$$\displaystyle=2\int_{\mathcal{X}}\frac{\det\left(\begin{pmatrix}f(x)^{\top}\\ g(x)^{\top}\end{pmatrix}\Sigma\bigl{(}f(x),g(x)\bigr{)}\right)^{\frac{1}{2}}_{2\times 2}}{f(x)^{\top}\Sigma f(x)}\,\mathrm{d}x,$$

$$\mathcal{M}=$$

$$=$$

$${\left\{\begin{array}{c}{t}_i\\ p_i\end{array}\right\}}_{i=1,2,3}=\left\{\begin{array}{ccc}-\frac{1}{3}& 0& \frac{1}{3}\\ \frac{1}{3}& \frac{1}{3}& \frac{1}{3}\end{array}\right\}\text{mbox}and\left\{\begin{array}{ccc}-\frac{1}{12}& 0& \frac{1}{12}\\ \frac{1}{3}& \frac{1}{3}& \frac{1}{3}\end{array}\right\}.$$

$$M(c)=(1-c)\left(\begin{array}{ccc}1& 0& 0\\ 0& 1& 0\\ 0& 0& 1\end{array}\right)+c\left(\begin{array}{ccc}1& 0& \frac{\sqrt{2}+\sqrt{6}}{3}\\ 0& \frac{1}{3}& 0\\ \frac{\sqrt{2}+\sqrt{6}}{3}& 0& \frac{5}{3}\end{array}\right).$$

$$f(x)=$$

$$=$$

$$f(x)=f_P(x)=(1,x,x^2,\dots ,x^{n-1}{)}^{\top }\in \mathbb{R}[x{]}^n.$$

$${\sigma }_P^2(x)=(1+x^2{)}^{n-1}.$$

$$f_F(t)=B{f}_P(x){\lambda }_0(x),{\lambda }_0(x)=1/(1+x^2{)}^{(n-1)/2}.$$

$$\sin (2\pi t)=\frac{2x}{1+x^2},\cos (2\pi t)=\frac{1-x^2}{1+x^2},\mathrm{d}t=\frac{\mathrm{d}x}{\pi (1+x^2)}.$$

$$\sin (2\pi kt)=$$

$$\cos (2\pi kt)=$$

$$\int_{-1/2}^{1/2}f_{F}(t)f_{F}(t)^{\top}\mathrm{d}t=B\biggl{(}\int_{-\infty}^{\infty}f_{P}(x)\lambda_{0}(x)^{2}f_{P}(x)^{\top}\frac{\mathrm{d}x}{\pi(1+x^{2})}\biggr{)}B^{\top}.$$

$$(B^{\top}B)^{-1}_{i,j}=\int_{-\infty}^{\infty}\frac{x^{i+j-2}}{\pi(1+x^{2})^{n}}\mathrm{d}x=\begin{cases}\displaystyle\frac{\Gamma\bigl{(}\frac{i+j-1}{2}\bigr{)}\Gamma\bigl{(}n-\frac{i+j-1}{2}\bigr{)}}{\pi\Gamma(n)}&(\mbox{$i+j$ is even}),\\ 0&(\mbox{$i+j$ is odd}),\end{cases}$$

$$B=\left(\begin{array}{ccc}1& 0& 1\\ 0& 2\sqrt{2}& 0\\ \sqrt{2}& 0& -\sqrt{2}\end{array}\right),\left(\begin{array}{cccc}1& 0& 1& 0\\ 0& 1& 0& 1\\ 1& 0& -3& 0\\ 0& 3& 0& -1\end{array}\right),$$

$$\mathcal{M}_{P}=\biggl{\{}\int_{-\infty}^{\infty}f_{P}(x)f_{P}(x)^{\top}\mathrm{d}\rho(x)\succ 0\mid\rho\in\mathcal{P}\biggr{\}}$$

$${\mathcal{M}}_F$$

$$\displaystyle=\bigl{\{}BMB^{\top}\mid M\in\mathcal{M}_{P}\bigr{\}}=B\mathcal{M}_{P}B^{\top}.$$

Full text

Key words: D-optimality, Fourier regression, Hankel matrix, Möbius group, volume-of-tube method, weighted polynomial regression.

1 Introduction

Suppose that we observe pairs of explanatory variables $x_{i}\in\mathcal{X}$ and response variables $y_{i}\in\mathbb{R}$, $i=1,\ldots,N$. Here, $\mathcal{X}\subset\mathbb{R}$ is a domain of explanatory variables, and typically, a segment of $\mathbb{R}$. For such data, we consider the regression model

[TABLE]

where $b=(b_{1},\ldots,b_{n})^{\top}$ is an unknown coefficient vector, and $f(x)=(f_{1}(x),\ldots,f_{n}(x))^{\top}$, $x\in\mathcal{X}$, is a piecewise smooth regression basis vector. For the problem of this paper, we assume that the variance function $\sigma^{2}(x)>0$ is known. When $\sigma^{2}(x)$ is not a constant, the regression model is called weighted.

For experimental design, it is assumed that the explanatory variables $x_{i}$ can be chosen arbitrarily within its domain $\mathcal{X}\subset\mathbb{R}$. The allocation of $\{x_{1},\ldots,x_{N}\}\subset\mathcal{X}$ to optimize some target function is called optimal experimental design. For example, for D-optimality, we take a function $\det(\Sigma)$ with $\Sigma=\mathrm{Var}(\widehat{b})$, where $\widehat{b}$ is the ordinary least square (OLS) estimator of $b$. Here,

[TABLE]

the information matrix.

Following Kiefer and Wolfowitz (1959), in an optimal design, allocation $\{x_{1},\ldots,x_{N}\}$ is regarded as the probability measure over $\mathcal{X}$ with mass $p_{i}=1/N$ at each point $x_{i}$. We write this discrete probability measure as

[TABLE]

Viewed from this point, the problem is formalized as that of optimization with respect to the probability measure over $\mathcal{X}$. Most criteria in the literature including the criterion mentioned above are convex or concave functionals of the probability measure, and can be considered in the framework of convex analysis (Wynn, 1985; Pukelsheim, 2006).

In this paper, we propose a new non-convex criterion based on simultaneous confidence bands. The pointwise confidence band is based on the confidence region for regressor $b^{\top}f(x)$ at a fixed point $x$. On the other hand, the simultaneous confidence band is the confidence region for the full regression curve $\{(x,b^{\top}f(x))\mid x\in\mathcal{X}\}\subset\mathbb{R}^{2}$. The standard form of the simultaneous confidence band of hyperbolic-type is of the form

[TABLE]

where $u\pm v$ stands for the region $(u-v,u+v)$. The threshold $c_{\alpha}$ is determined so that the event (1) holds for all $x\in\mathcal{X}$ with given probability $1-\alpha$ (Working and Hotelling, 1929; Scheffé, 1959; Wynn and Bloomfield, 1971; Liu, 2010). The simultaneous confidence bands are useful when $x$ cannot be determined in advance. (See van Dyk (2014) for an application in experimental particle physics.) As shown in the next section, $c_{\alpha}$ is also a functional of the allocation $\{x_{1},\ldots,x_{N}\}$, and hence, we can consider an optimal design that in some way minimizes both the threshold $c_{\alpha}$ and $f(x)^{\top}\Sigma f(x)$. In fact, from the general equivalence theorem of Kiefer and Wolfowitz (1959), the design measure that minimizes $\max_{x\in\mathcal{X}}f(x)^{\top}\Sigma f(x)$ coincides with the D-optimal design. Therefore, we propose the use of $c_{\alpha}$ as a criterion of optimal design, and consider the corresponding optimal design as the tube-volume optimal (TV-optimal) design. If a design is optimal under both the tube-volume criterion and the D-criterion, it becomes the universal optimal design to minimize the width of confidence bands (1).

From its definition, $c_{\alpha}$ is a complicated function of $\Sigma$. However, when $\alpha$ is small, $c_{\alpha}$ tends to a simpler function. This approximation is due to the volume-of-tube method used to construct simultaneous confidence bands in curvilinear regression curves (Naiman, 1986; Johansen and Johnstone, 1990; Sun and Loader, 1994; Lu and Kuriki, 2017). The volume-of-tube method is a methodology to approximate the probability of the maximum of a Gaussian random field (Sun, 1993; Kuriki and Takemura, 2001, 2009; Takemura and Kuriki, 2002; Adler and Taylor, 2007). As shown later, $c_{\alpha}$ corresponds to the upper tail probability of the maximum of a Gaussian field, and hence, the volume-of-tube method works well.

As concrete regression models, weighted polynomial and Fourier regressions are mainly covered here. In these models, we will see that there is a group referred to as the Möbius transform that keeps the tube-volume optimal design problem invariant. In general, a group action simplifies problems. (See Section 13 of Pukelsheim (2006) for invariant optimal experimental design.) The use of such group invariance is another subject of this paper.

The optimal design problem focusing on the width of the simultaneous confidence bands can be formalized in a different way. When comparing two estimated regression curves, Dette and Schorning (2016) and Dette, et al. (2017) proposed to minimize the $L_{p}$- or $L_{\infty}$-norm of the variance function of the estimator of the difference between the two curves. They demonstrated that their proposal reduces the width substantially compared with the pair of optimized designs for individual regression models. Different from them, our objective function is the width $c_{\alpha}$ of the simultaneous confidence band standardized by standard deviation.

The outline of this paper is as follows. Section 2 summarizes the volume-of-tube formula to construct approximate simultaneous confidence bands, and formalizes the tube-volume criterion and the corresponding optimal design. Section 3 analyzes the tube-volume optimal designs for Fourier and weighted polynomial regressions. The Möbius group is proved to keep the optimization problem invariant, and hence can be used to reduce the dimension of the problem. Using this consideration, Section 4 identifies the tube-volume optimal design in the weighted polynomial regression and the Fourier regression when $n=3$. Some proofs are given in Appendix.

2 Tube-volume optimal design

2.1 Volume-of-tube formula for simultaneous confidence bands

In this subsection, we briefly summarize the volume-of-tube method. This is a general methodology used to approximate the probability of the maximum of a smooth Gaussian random process or random field. Here, we describe how this method is used to determine threshold $c_{\alpha}$.

As mentioned in Section 1, threshold $c_{\alpha}$ should be determined as a solution $c=c_{\alpha}$ of

[TABLE]

where $\Sigma^{\frac{1}{2}}$ is a symmetric square-root matrix such that $(\Sigma^{\frac{1}{2}})^{2}=\Sigma$.

We define the normalized basis vector and its trajectory as

[TABLE]

respectively. From this definition, the trajectory is a subset of the ($n-1$)-dimensional unit sphere:

[TABLE]

In particular, when $\mathcal{X}$ is a segment, $\gamma_{\Sigma}$ is a curve on the unit sphere. Let $\mathrm{Vol}_{1}(\cdot)$ denote the one-dimensional volume, that is, the length. Then, when $c$ is large, the volume-of-tube method provides an approximation to the upper tail probability of the maximum in (2). Further, let $\chi^{2}_{\nu}$ denote the chi-square random variable with $\nu$ degrees of freedom.

Proposition 2.1**.**

(i) As $c\to\infty$,

[TABLE]

(ii) For all $c>0$, the left-hand side of (4) is bounded above by

[TABLE]

where $\chi(\gamma_{\Sigma})$ is the number of the connected components of the set $\gamma_{\Sigma}\,(\subset\mathbb{S}^{n-1})$ provided that any connected component of $\gamma_{\Sigma}$ is not a closed curve.

If we admit approximation (4), an approximate threshold $c_{\alpha}$ can be determined from the equation

[TABLE]

This means that the smaller the value of $\mathrm{Vol}_{1}(\gamma_{\Sigma})$, the smaller is $c_{\alpha}$.

The statement (ii) above is due to Naiman (1986). Alternative proofs of the inequality can be found in Johnstone and Siegmund (1989) and Takemura and Kuriki (2002). See Lu and Kuriki (2017) for a generalization of Naiman’s inequality. By equating (5) to be $\alpha$, we have a conservative threshold for the simultaneous confidence band.

The volume-of-tube method in Proposition 2.1 is based on the property that the normalized vector $\eta=\xi/\|\xi\|$, $\xi=\Sigma^{-\frac{1}{2}}(\widehat{b}-b)$, is distributed uniformly on the unit sphere $\mathbb{S}^{n-1}$, independently of its length $\|\xi\|$. This occurs when the observation error $(\varepsilon_{1},\ldots,\varepsilon_{n})^{\top}$ is distributed as the elliptically contoured distribution, and hence the generalized least square estimator $\widehat{b}$ follows the elliptically contoured distribution as well (see, e.g., Theorem 2.6.3 of Fang and Zhang (1990)). Part (ii) of Proposition 2.1 still holds as follows.

Proposition 2.2**.**

Suppose that $\xi=(\xi_{1},\ldots,\xi_{n})^{\top}=\Sigma^{-\frac{1}{2}}(\widehat{b}-b)$ is distributed according to an elliptically contoured distribution with mean zero and an identity covariance matrix. Then, for all $c>0$,

[TABLE]

where $R_{k}^{2}=\sum_{i=1}^{k}\xi_{i}^{2}$, provided that any connected component of $\gamma_{\Sigma}$ is not a closed curve.

The proof is given in Appendix A.1. Part (i) of Proposition 2.1 does not hold in general. It depends on the tail behavior of the elliptically contoured distribution.

2.2 Tube-volume criterion

From (5), we find that the smaller the value of $\mathrm{Vol}_{1}(\gamma_{\Sigma})$, the narrower is the width of the confidence band. In this subsection, we formalize the experimental design optimization problem of the allocation of explanatory variables to minimize $\mathrm{Vol}_{1}(\gamma_{\Sigma})$.

Here, we give our assumptions on $f(x)$.

Assumption 2.3**.**

$f:\mathcal{X}\to\mathbb{R}^{n}$* is a continuous and piecewise $C^{1}$-function. Image $f(\mathcal{X})$ spans $\mathbb{R}^{n}$.*

From elementary geometry, the volume of $\gamma_{\Sigma}$ in (3) is given by

[TABLE]

where $g(x)=\mathrm{d}f(x)/\mathrm{d}x$. We call (7) the tube-volume (TV) criterion.

Note that (7) is invariant with respect to scale $\Sigma\mapsto k\Sigma$ ($k>0$). Because of this, we introduce the set of all nonnegative finite measures (not necessarily probability measures) on $\mathcal{X}$ denoted by $\mathcal{P}$. Each element of $\rho\in\mathcal{P}$ corresponds to an experimental design. This is an extension of the design measure of Kiefer and Wolfowitz (1959). We also extend the set of moment matrices. Thus, the set of all non-singular information matrices is denoted by

[TABLE]

where “$\succ 0$” denotes positive definiteness. From its definition, $\mathcal{M}$ forms a convex cone. Our optimal design problem is formulated as minimizing $\mathrm{Vol}_{1}(\gamma_{\Sigma})$ in (7) subject to $\Sigma^{-1}\in\mathcal{M}$. The lemma below is a direct consequence of this invariance.

Lemma 2.4**.**

The design $\left\{\begin{matrix}x_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq N}$ with variance function $\sigma_{1}^{2}(x)$, and the design $\left\{\begin{matrix}x_{i}\\ q_{i}\end{matrix}\right\}_{1\leq i\leq N}$ with variance function $\sigma_{2}^{2}(x)$, $q_{i}=kp_{i}\sigma_{2}^{2}(x_{i})/\sigma_{1}^{2}(x_{i})$, give the same volume, where $k>0$ is a constant so that $\sum_{i}q_{i}=1$.

Proof.

The information matrices $M_{1}$ and $M_{2}$ of the two designs satisfy $M_{1}=kM_{2}$. ∎

A difficulty with this optimization problem is that this is not a convex problem. Figure 1 depicts the volume $\mathrm{Vol}_{1}(\gamma_{M(c)^{-1}})$ for a mixing design connecting two Fourier designs with three element bases ((8) with $n=3$) with weights $1-c$ and $c$ :

[TABLE]

The information matrix is

[TABLE]

We see that $\mathrm{Vol}_{1}(\gamma_{M^{-1}(c)})$ is not convex in $c$.

3 Tube-volume optimal design for polynomial and Fourier regressions

3.1 Equivalence between weighted polynomial regression and Fourier regression

From this section, we focus on Fourier regression and weighted polynomial regression. In linear optimal design theory, the Fourier regression model has been used as one of the standard models to see the performance of the proposed criteria. The weighted polynomial introduced here has the same mathematical structure as the Fourier regression and will be used to analyze it.

The Fourier (trigonometric) regression has basis vector

[TABLE]

defined on the domain $\mathcal{X}=(-1/2,1/2]$. For the Fourier regression, we only deal with the constant variance $\sigma^{2}(x)=\sigma_{F}^{2}(x)\equiv 1$. Although the Fourier regression is not used for even values of $n$ in practice, we define it for the sake of consistency.

The polynomial regression is a regression model with basis vector

[TABLE]

Here, we set the domain $\mathcal{X}$ to be the whole real line $\mathbb{R}$. The case where $\mathcal{X}$ is an infinite interval will be briefly discussed in Section 5. For the polynomial regression, we assume the variance function of form $\sigma^{2}(x)=Q(x)^{n-1}$, where $Q(x)$ is an arbitrary positive quadratic function. As a canonical form of this class of variance functions, we use

[TABLE]

Later, we introduce a parameterization for $Q(x)$ (see (27)).

In this subsection, we see that under the tube-volume criterion, the optimization problem for the Fourier regression is equivalent to that for the weighted polynomial regression. That is, the optimization problem in the Fourier regression can be translated to one in the weighted polynomial regression, and vice versa.

The model we discuss is a special case of the model proposed by Dette, et al. (1999), Section 2.2, who study the D-optimality. Further, Dette and Melas (2003) make use of the connection between the weighted polynomial and Fourier regressions. We will return briefly to question on the D-optimality in Section 3.5 below.

From the lemma below, the transformation $x=\tan(\pi t)$ connects the two regression models.

Lemma 3.1**.**

There exists an $n\times n$ non-singular matrix $B$ such that for all $x\in\mathbb{R}$ and $t\in(-1/2,1/2]$ satisfying $x=\tan(\pi t)$, we have

[TABLE]

Proof.

Note that

[TABLE]

It is known that

[TABLE]

(Moriguti, et al., 1957, pages 186–187). Substituting the formulas for $\sin(2\pi t)$ and $\cos(2\pi t)$ in (12) and expressing $f_{F}(t)$ as a rational function in $x$, we have formula (11).

To prove that $B$ is non-singular, consider the integral

[TABLE]

Here, we used (12). The left-hand side is the identity matrix $I_{n}$ by standard orthogonality. Hence, it is enough to check that the integrals in the parentheses of the right-hand side exist. The matrix in the parentheses of the right-hand side of (13) is $(B^{\top}B)^{-1}$ with $(i,j)$ element

[TABLE]

which exists for $i,j\leq n$. Hence, $B$ is non-singular. ∎

Lemma 3.1 means that the Fourier regression model $y_{i}=b^{\top}f_{F}(t_{i})+\varepsilon_{i}$, $\varepsilon_{i}\sim N(0,1)$, is rewritten as the weighted polynomial model $\widetilde{y}_{i}=\widetilde{b}^{\top}f_{P}(x_{i})+\widetilde{\varepsilon}_{i}$, by letting $x_{i}=\tan(\pi t_{i})$, $\widetilde{y}_{i}=\lambda_{0}(x_{i})^{-1}y_{i}$, $\widetilde{b}=B^{\top}b$, and $\widetilde{\varepsilon}_{i}\sim N(0,\lambda_{0}(x_{i})^{-2})$.

When $n=3,4$,

[TABLE]

respectively.

The set of information matrices for the polynomial regression

[TABLE]

is referred to as the moment cone (Karlin and Studden, 1966). The set of information matrices for the Fourier regression is given by

[TABLE]

The following lemma gives the equivalence of the Fourier regression and the polynomial regression as the optimization problem for the tube-volume criterion.

Theorem 3.2**.**

Let $\mathrm{Vol}_{F}(\gamma_{M^{-1}})$ and $\mathrm{Vol}_{P}(\gamma_{M^{-1}})$ be the length of $\gamma_{M^{-1}}$ given in (7) with $f(x)$ being $f_{F}(x)$ in (8), and $f_{P}(x)$ in (9), respectively. Then, it holds that $\mathrm{Vol}_{P}(\gamma_{M^{-1}})=\mathrm{Vol}_{F}(\gamma_{(BMB^{\top})^{-1}})$.

Proof.

The derivatives of $f_{F}(t)$ and $f_{P}(x)$ are denoted by $g_{F}(t)=\mathrm{d}f_{F}(t)/\mathrm{d}t$ and $g_{P}(x)=\mathrm{d}f_{P}(x)/\mathrm{d}x$, respectively. Then,

[TABLE]

Therefore,

[TABLE]

[TABLE]

and

[TABLE]

∎

Theorem 3.2 and (16) imply that

[TABLE]

That is, the optimization problems for the polynomial regression and the Fourier regression are mathematically equivalent. For example, the information matrix $M=\sum_{i}f_{F}(t_{i})f_{F}(t_{i})^{\top}p_{i}\in\mathcal{M}_{F}$ for the Fourier regression, and the information matrix for the polynomial regression

[TABLE]

give the same volume.

This equivalence is stated in terms of design measure as follows.

Theorem 3.3**.**

The design $\left\{\begin{matrix}t_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq N}$ for the Fourier regression, and the design $\left\{\begin{matrix}x_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq N}$, $x_{i}=\tan(\pi t_{i})$, for the weighted polynomial regression with variance function $\sigma_{P}^{2}(x)$ give the same volume. If the former is tube-volume optimal in the Fourier regression, then so is the latter in the polynomial regression with variance $\sigma_{P}^{2}(x)$, and vice versa.

In this paper, the (discrete) uniform designs in the Fourier regression and their counterparts in the polynomial regression play important roles. It is known that, in the Fourier regression, the uniform design in which $x_{i}$ are allocated as equally spaced with equal weights is D-optimal (Guest, 1958). Because of the symmetry, it is conjectured that the uniform design is the tube-volume optimal design as well. In Section 4, we prove that this is true for $n=3$, and conjecture that it is true for all $n$.

The $n$-point discrete uniform design for the Fourier regression symmetric about the origin is

[TABLE]

For later use, we provide the concrete forms of the information matrix $M$ for the weighted polynomial designs with $\sigma^{2}(x)=\sigma_{P}^{2}(x)$ in (10),

[TABLE]

Lemma 3.4**.**

The information matrix $M=(M_{i,j})$ of the weighted polynomial design (18) scaled such that $M_{1,1}=1$ is given by

[TABLE]

For the proof, see Appendix A.2. When $n=3$ and $4$,

[TABLE]

respectively.

The key transform connecting Fourier and polynomial regressions was the tangent transform $x=\tan(\pi t)$. For the same purpose, generalized transforms $x=q\tan(\pi(t-\theta))+r$, $q\neq 0$, can be used. This is a composite map of the tangent transform and the Möbius transform to be discussed below.

3.2 The Möbius group action on the moment cone

In this subsection, we introduce the Möbius group (transformation) acting on the set of design measures and the set of information matrices in polynomial regression. We will show that the Möbius group action reduces the dimension of the minimization problem for the tube-volume criterion. For a recent paper in which the Möbius transformation acts on polynomials, see Mackey, et al (2015).

The real Möbius transformation is defined on the extended real numbers $\overline{\mathbb{R}}=\mathbb{R}\cup\{\pm\infty\}$ as follows:

[TABLE]

Here, we assume that

[TABLE]

This forms a group with product

[TABLE]

The inverse is $\varphi^{-1}(\cdot;a,b,c,d)=\varphi(\cdot;d,-b,-c,a)$. The identity element is $e=\varphi(\cdot;a,0,0,a)$, $a\neq 0$. This is a subgroup of the complex Möbius group referred to as projective general linear group $\mathrm{PGL}(2,\mathbb{C})$.

Now, let $f_{P}(x)=(1,x,\ldots,x^{n-1})^{\top}$ be the polynomial basis. We define an $n\times n$ matrix $A=A(a,b,c,d)$ as

[TABLE]

We write the factor $\lambda$ as $\lambda(x;a,b,c,d)$ instead of $\lambda(x;c,d)$ to clarify that this is an invariant function under the group action (20) in the sense that

[TABLE]

Proposition 3.5**.**

The $(i,j)$ element of $A=A(a,b,c,d)$ is

[TABLE]

The proof is straightforward and omitted. When $n=3$ and $n=4$,

[TABLE]

respectively.

The set

[TABLE]

is a representation of general linear group $\mathrm{GL}(2,\mathbb{R})$ and hence forms a group (Gross and Holman, 1980). The proof of the proposition below is straightforward and omitted.

Proposition 3.6**.**

Set $\mathcal{A}$ forms a matrix algebraic group. The identity matrix is $A(1,0,0,\allowbreak 1)=(-1)^{n-1}A(-1,0,0,-1)=I_{n}$, and the inverse of $A=A(a,b,c,d)$ is given by $(ad-bc)^{-(n-1)}A(d,-b,-c,a)$.

Proposition 3.7**.**

For $A=A(a,b,c,d)\in\mathcal{A}$, $\det(A)=(ad-bc)^{n(n-1)/2}$.

Proof.

For different $x_{1},\ldots,x_{n}$, we have

[TABLE]

By taking determinants,

[TABLE]

Substituting

[TABLE]

into (24) yields $\det(A)=(ad-bc)^{n(n-1)/2}$. ∎

Proposition 3.8**.**

The Möbius group is reparameterized as

[TABLE]

where $\sqcup$ is the disjoint union. Group $\mathcal{A}$ is reparameterized as

[TABLE]

$\det A(q,r,0,1)=q^{n(n-1)/2}$* and $\det A(\pm s,\mp t,t,s)=\pm 1$ for $s^{2}+t^{2}=1$.*

Proof.

We have the following relations:

[TABLE]

and

[TABLE]

The results in the proposition follow by letting $k=(c^{2}+d^{2})^{(n-1)/2}$,

[TABLE]

∎

The sets of transformations $\{\varphi(\cdot;\pm s,\mp t,t,s)\mid s^{2}+t^{2}=1\}$ and $\{\varphi(\cdot;q,r,0,1)\mid q\neq 0\}$ form subgroups of the Möbius group, which are isomorphic to the orthogonal group $\mathrm{O}(2,\mathbb{R})$ and the affine group acting on $\mathbb{R}$, respectively.

Theorem 3.9**.**

Let $A\in\mathcal{A}$ and $M\in\mathcal{M}_{P}$ be $n\times n$ matrices. Then, $AMA^{\top}\in\mathcal{M}_{P}$. Moreover,

[TABLE]

That is, group $\mathcal{A}$ acts on the moment cone $\mathcal{M}_{P}$.

Proof.

Suppose that $M=\sum_{i}f_{P}(x_{i})f_{P}(x_{i})^{\top}w_{i}$. Let $y_{i}=\varphi(x_{i};a,b,c,d)$. Note that $\lambda(y;d,-b,-c,a)\lambda(x;a,b,c,d)=(ad-bc)^{-(n-1)}$. Then,

[TABLE]

where

[TABLE]

Therefore, $A\mathcal{M}_{P}A^{\top}\subset\mathcal{M}_{P}$. Because $\mathcal{A}$ is a group, $A\mathcal{M}_{P}A^{\top}=\mathcal{M}_{P}$. ∎

The Möbius group action on the polynomial basis $f(x)$ has been introduced by (21). Similarly, we define the Möbius group action on the variance function $\sigma^{2}(x)=Q(x)^{n-1}$. This provides a parameterization for the variance function.

Using $\sigma_{P}^{2}(x)=(1+x^{2})^{n-1}$ in (10), for $ad-bc\neq 0$, we define

[TABLE]

or

[TABLE]

Note that $\sigma_{P}^{2}(x)=\sigma_{P}^{2}(x;1,0,0,1)$. This is always positive because of $ad-bc\neq 0$. For

[TABLE]

as well as (22), we have

[TABLE]

The parameterization (27) with $(a,b,c,d)$ is redundant, since $Q(x)$ has only three parameters. The lemma below shows that the stabilizer keeping the variance $\sigma_{P}^{2}(\cdot;a,b,c,d)$ invariant is the orthogonal subgroup with dimension one.

Lemma 3.10**.**

$\sigma_{P}^{2}(\cdot;a,b,c,d)=\sigma_{P}^{2}(\cdot;a^{\prime},b^{\prime},c^{\prime},d^{\prime})$* if and only if there exist $s,t$, $s^{2}+t^{2}=1$ such that*

[TABLE]

Proof.

By direct calculations,

[TABLE]

On the other hand, “$\sigma_{P}^{2}(\cdot;a_{0},b_{0},c_{0},d_{0})=\sigma_{P}^{2}(\cdot;1,0,0,1)$” $\Leftrightarrow$ “$\sigma_{P}^{2}(\varphi(\cdot;a,b,c,d);a_{0},b_{0},c_{0},d_{0})\allowbreak=\sigma_{P}^{2}(\varphi(\cdot;a,b,c,d);1,0,0,1)$” $\Leftrightarrow$ “$\sigma_{P}^{2}(x;a^{\prime},b^{\prime},c^{\prime},d^{\prime})=\sigma_{P}^{2}(x;a,b,c,d)$ with $\varphi(\cdot;a^{\prime},b^{\prime},c^{\prime},d^{\prime})=\varphi(\cdot;a_{0},b_{0},c_{0},d_{0})\circ\varphi(\cdot;a,b,c,d)$” by (28). ∎

Remark 3.11**.**

When $n=2$, $|ad-bc|/\{\pi\sigma_{P}^{2}(x;a,b,c,d)\}$ is a probability density of Cauchy distribution family on $x\in\mathbb{R}$. Noting that $\dot{\varphi}(x;a,b,c,d)=(ad-bc)\lambda(x;a,b,c,d)^{2}$, and that $a^{\prime}d^{\prime}-b^{\prime}c^{\prime}=(ad-bc)(a_{0}d_{0}-b_{0}c_{0})$, (28) reads

[TABLE]

This means that the Cauchy distribution family is closed under the Möbius transform (McCullagh, 1996). See also Kato and McCullagh (2014) for Cauchy families in directional statistics.

3.3 Canonical parameterizations for information matrices

As we have shown in Section 2.1, the optimal design problem is optimization with respect to matrix $M$ over the set of information matrices $\mathcal{M}$. Here, $M=\int_{\mathcal{X}}f(x)f(x)^{\top}\frac{1}{\sigma^{2}(x)}\mathrm{d}\rho(x)\in\mathcal{M}$ and the design measure $\rho\in\mathcal{P}$ is one-to-many. For the sake of optimization, we need to parameterize the set $\mathcal{M}$.

We first consider $\mathcal{M}_{P}$ in (15) for the polynomial regression, and then interpret the results in terms of $\mathcal{M}_{F}$ in (16) for the Fourier regression.

The structure of the moment cone $\mathcal{M}_{P}$ is well-studied in the context of the classical moment problem. One canonical parameterization for $\mathcal{M}_{P}$ is given in Chapter II, Section 3 of Karlin and Studden (1966). The statement is summarized in Proposition 3.1 of Kato and Kuriki (2013).

Proposition 3.12**.**

$M\in\mathcal{M}_{P}$* is uniquely represented with $2n-1$ parameters $(w_{0},\ldots,w_{n-1},\allowbreak x_{1},\ldots,x_{n-1})$ as*

[TABLE]

where we let $f_{P}(\pm\infty)=(0,\ldots,0,1)^{\top}$.

Note that $f_{P}(\pm\infty)=\lim_{x\to\pm\infty}f_{P}(x)/x^{n-1}=\lim_{x\to\pm\infty}f_{P}(x)/(1+x^{2})^{(n-1)/2}$.

Let $x_{0}=\pm\infty$. From the same argument of the proof of Theorem 3.9, by considering the Möbius transform $x_{i}\mapsto(ax_{i}+b)/(cx_{i}+d)$, $i=0,1,\ldots,n-1$, we find that the fixed point $x_{0}=\pm\infty$ in (29) can be moved to an arbitrary point in $\mathbb{R}$.

Theorem 3.13**.**

Let $x_{0}\in\mathbb{R}\cup\{\pm\infty\}$ be fixed arbitrarily. $M\in\mathcal{M}_{P}$ is uniquely represented with $2n-1$ parameters $(w_{0},\ldots,w_{n-1},x_{1},\ldots,x_{n-1})$ as

[TABLE]

where we assume that $f_{P}(\pm\infty)=(0,\ldots,0,1)^{\top}$.

The counterpart for the moment cone (16) for trigonometric functions is obtained using Lemma 3.1.

Theorem 3.14**.**

Let $t_{0}\in(-\frac{1}{2},\frac{1}{2}]$ be fixed arbitrarily. $M\in\mathcal{M}_{F}$ is uniquely represented with $2n-1$ parameters $(w_{0},\ldots,w_{n-1},t_{1},\ldots,t_{n-1})$ as

[TABLE]

A square matrix $M=(m_{i,j})$ is said to be Hankel if $m_{i,j}=m_{k,l}$ when $i+j=k+l$. For example, matrices $M$ in (19) and (31) are Hankel. Obviously, each $M\in\mathcal{M}_{P}$ should be an $n\times n$ positive definite Hankel matrix. It is known that the converse is also true.

Proposition 3.15**.**

The moment cone $\mathcal{M}_{P}$ in (15) is characterized as

[TABLE]

For the proof, see (9.1) of Karlin and Studden (1966), p. 199. This also gives a unique representation of $\mathcal{M}_{P}$ with $2n-1$ parameters $(m_{0},m_{1},\ldots,m_{2n-2})$.

Theorem 3.9 combined with Proposition 3.15 implies that group $\mathcal{A}$ acts on the cone of (positive definite) Hankel matrices. For the Möbius group action on Hankel matrices, see also Heinig and Rost (1989, 2010).

3.4 Invariance under the Möbius group

In this subsection, we consider the polynomial regression. We formalized our optimal experimental design problem to find the minimizer $M\in\mathcal{M}_{P}$ of $\mathrm{Vol}_{1}(\gamma_{M^{-1}})$ in (7).

Theorem 3.16**.**

For $M\in\mathcal{M}_{P}$ and $A\in\mathcal{A}$,

[TABLE]

Theorem 3.16 and Theorem 3.9 imply that the minimizer of $\mathrm{Vol}_{1}(\gamma_{M^{-1}})$ with respect to $M\in\mathcal{M}_{P}$ forms an orbit (or a union of orbits) on $\mathcal{M}_{P}$.

Proof.

Let $y=\varphi(x)=\varphi(x;a,b,c,d)=(ax+b)/(cx+d)$. Then, $f_{P}(y)=f_{P}(\varphi(x))=Af_{P}(x)\lambda(x)$, where $A=A(a,b,c,d)$, $\lambda(x)=\lambda(x;a,b,c,d)$. Taking derivatives with respect to $x$,

[TABLE]

Therefore,

[TABLE]

and

[TABLE]

By combining this with

[TABLE]

and $\mathrm{d}y=\dot{\varphi}(x)\mathrm{d}x$, we have

[TABLE]

∎

Theorem 3.17**.**

The volumes of the weighted polynomial design $\left\{\begin{matrix}x_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq N}$ with variance $\sigma_{P}^{2}(x;a_{0},b_{0},c_{0},d_{0})$, and the design $\left\{\begin{matrix}y_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq N}$, $y_{i}=\varphi(x_{i};a,b,c,d)$ with variance $\sigma_{P}^{2}(y;a^{\prime},b^{\prime},c^{\prime},d^{\prime})$ are the same, where $(a^{\prime},b^{\prime},c^{\prime},d^{\prime})$ is determined by

[TABLE]

Proof.

In (25) of the proof of Theorem 3.9, let $w_{i}=p_{i}/\sigma_{P}^{2}(x_{i};a_{0},b_{0},c_{0},d_{0})$. Then, by (28),

[TABLE]

This means that information matrices $M_{1}$ and $M_{2}$ of the two designs satisfy $M_{1}=AM_{2}A^{\top}$ and hence have the same volume by Theorem 3.16. ∎

3.5 D-optimal design for weighted polynomial regression

We characterize the D-optimal design for the weighted polynomial regression as an orbit of the Möbius group action. We start from the fact that in the Fourier regression, the uniform design is D-optimal.

Proposition 3.18** (Guest (1958)).**

In the Fourier regression with the basis (8), among the $n$-point discrete design, only the uniform design

[TABLE]

is D-optimal, where \theta\in\bigl{(}-\frac{1}{2n},\frac{1}{2n}\bigr{)} is an arbitrary constant. The information matrix at the optimal point is the identity $I_{n}$.

Let $M_{F}$ be an information matrix of a Fourier design $\left\{\begin{matrix}t_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq n}$. By making a change of variables $y_{i}=\tan(\pi t_{i})$ and $y_{i}=\varphi(x_{i};a,b,c,d)$, we have from (11), (21), and (26) that

[TABLE]

where

[TABLE]

is the information matrix of the design $\left\{\begin{matrix}x_{i}\\ p_{i}\end{matrix}\right\}_{1\leq i\leq n}$ for the weighted polynomial regression with variance function $\sigma_{P}^{2}(x;a,b,c,d)$. Because $\det(M_{F})=\det(AB)^{2}\times\det(M_{P})$, the D-optimal problem for searching optimal $t_{i}$ and $p_{i}$ in the Fourier regression are equivalent to searching for optimal $x_{i}$ and $p_{i}$ in the weighted polynomial regression. Hence, Proposition 3.18 is translated into the weighted polynomial regression as follows.

Theorem 3.19**.**

In the weighted polynomial regression of degree $n-1$ with variance $\sigma_{P}^{2}(x;a_{0},b_{0},c_{0},d_{0})$, among the $n$-point discrete design, only the design $\left\{\begin{matrix}x_{i}\\ \frac{1}{n}\end{matrix}\right\}_{1\leq i\leq n}$ with $x_{i}=\varphi^{-1}(\varphi(\tan(\pi t^{0}_{i});s,-t,t,s);a_{0},b_{0},c_{0},d_{0}))$ is D-optimal, where $t^{0}_{i}$ is given in (30), and $s,t$ are arbitrary numbers such that $s^{2}+t^{2}=1$. The information matrix at the D-optimal point is $A_{0}^{-1}(B^{\top}B)^{-1}(A_{0}^{\top})^{-1}$, where $A_{0}=A(a_{0},b_{0},c_{0},d_{0})$, and $(B^{\top}B)^{-1}$ is given in (14).

Proof.

Note that

[TABLE]

where $s=\cos(\pi\theta)$, $t=\sin(\pi\theta)$. $x_{i}=\varphi^{-1}(y_{i};a_{0},b_{0},c_{0},d_{0})$. ∎

When $(a_{0},b_{0},c_{0},d_{0})=(1,0,0,1)$, Theorem 3.19 reduces to Theorem 3.3 of Dette, et al. (1999).

4 Tube-volume optimal design for $n=3$

In the previous section, we discussed the Fourier regression and the polynomial regression having the basis of (8) and (9), respectively, of a general dimension $n$. In this section, we treat the case $n=3$. This is the simplest non-trivial case, because when $n=2$, $\mathrm{Vol}_{1}(\gamma_{M^{-1}})=2\pi$ irrespective of $M$.

When $n=3$, the problem is reduced to the minimization of

[TABLE]

where

[TABLE]

with respect to

[TABLE]

The volume becomes an elliptic integral, which does not have an explicit expression in general. Moreover, the number of parameters to be optimized is four. (Note that the integrand $\sqrt{h_{1}(x)}/h_{0}(x)$ is a homogeneous function in $m_{0},\ldots,m_{4}$). We will solve this minimization problem using the Möbius invariance.

4.1 Orbital decomposition

The Möbius group action defines an equivalence relation on the moment cone $\mathcal{M}_{P}$. We define $M_{0}\sim M_{1}$ for $M_{0},M_{1}\in\mathcal{M}_{P}$ if $M_{1}=AM_{0}A^{\top}$ for some $A\in\mathcal{A}$. The orbit passing through $M$ is denoted by

[TABLE]

The goal of this subsection is to provide the orbital decomposition of the polynomial moment cone $\mathcal{M}_{P}$. Let

[TABLE]

Theorem 4.1**.**

[TABLE]

where $\sqcup$ is the disjoint union.

Proof.

This is a consequence of Lemmas 4.2 and 4.3 below. ∎

Lemma 4.2**.**

For any $M\in\mathcal{M}_{P}$, there exist $0<v<1$ and $A\in\mathcal{A}$ such that $AMA^{\top}=M_{v}$.

Lemma 4.3**.**

$M_{v^{\prime}}\sim M_{v}$* if and only if $v^{\prime}=v$ or $v^{\prime}=(1-v)/(1+3v)$.*

The proofs of Lemmas 4.2 and 4.3 are given in Appendix A.3 and A.4, respectively. Note that the map

[TABLE]

defines a one-to-one correspondence between $(1,1/3)$ and $(1/3,1)$, and $v=1/3$ is the fixed point of this map.

The stabilizer of $\mathcal{A}$ at $M\in\mathcal{M}_{P}$ is defined as

[TABLE]

This is a subgroup of $\mathcal{A}$.

Theorem 4.4**.**

When $v\neq 1/3$,

[TABLE]

and when $v=1/3$,

[TABLE]

In particular,

[TABLE]

Proof.

The proof follows from the proof of Lemma 4.3. The details are omitted. ∎

Theorem 4.5**.**

The dimension of orbit $\mathcal{A}(M_{v})$ passing through $M_{v}$ is

[TABLE]

Proof.

Let $M=(m_{i+j-2})_{i,j=1,2,3}$ and $\widetilde{M}=(\widetilde{m}_{i+j-2})_{i,j=1,2,3}$ be $3\times 3$ Hankel matrices, and let $A=A(a,b,c,d)$ ($3\times 3$ matrix in (23)). Assume that $\widetilde{M}=AMA^{\top}$. Picking up the $(1,1)$, $(1,2)$, $(1,3)$, $(2,3)$, and $(3,3)$ elements and rearranging them, we have $\widetilde{m}=Fm$, where $m=(m_{0},m_{1},m_{2},m_{3},m_{4})^{\top}$, $\widetilde{m}=(\widetilde{m}_{0},\widetilde{m}_{1},\widetilde{m}_{2},\widetilde{m}_{3},\widetilde{m}_{4})^{\top}$, and

[TABLE]

Let $m_{v}=(1,0,v,0,1)^{\top}$. The tangent space of the orbit at $M_{v}$ is spanned by the four column vectors of

[TABLE]

The rank of this matrix is 4 when $v\neq 1/3$ and 3 when $v=1/3$. ∎

From Theorems 4.4 and 4.5, we see that $\dim\mathcal{A}_{M_{v}}+\dim\mathcal{A}(M_{v})=4=\dim\mathcal{A}$ as expected (e.g., Kawakubo (1992)).

4.2 Minimization over cross-section

From the orbital decomposition (Theorem 4.1) and the invariance of the volume on an orbit (Theorem 3.16), the optimization problem is reduced to the minimization of $\mathrm{Vol}_{1}(\gamma_{M_{v}^{-1}})$ with respect to $v$, where $M_{v}$ is defined in (32). The range of $v$ is taken to be $(0,\frac{1}{3}]$ or $[\frac{1}{3},1)$. We write $\mathrm{Vol}_{1}(\gamma_{M_{v}^{-1}})=\mathrm{len}(v)$ shortly. From the definition (7),

[TABLE]

where

[TABLE]

Note that $\mathrm{len}(v)$ is an elliptic integral. The following is the main theorem of this section.

Theorem 4.6**.**

The minimizer of $\mathrm{Vol}_{1}(\gamma_{M^{-1}})$ in (7) over $M\in\mathcal{M}_{P}$ is given if and only if $M$ is in the orbit

[TABLE]

The minimum volume is $4\pi\sqrt{2/3}$.

Proof.

Because of Theorem 4.1, it is enough to take the range $v\in(0,\frac{1}{3}]$. We use the inequality

[TABLE]

The equality holds iff $z=0$. Noting that

[TABLE]

$s(x;v)$ is bounded below by

[TABLE]

Therefore, $\mathrm{len}(v)$ is bounded below by

[TABLE]

This integral can be evaluated by counting the residues. When $v<1/3$, the poles are

[TABLE]

[TABLE]

and $\pm ix_{0}=\pm i$.

Denote the residues for $+ix_{1}$, $+ix_{2}$, and $+ix_{0}$ by $\mathrm{Res}(+ix_{1})$, $\mathrm{Res}(+ix_{2})$, and $\mathrm{Res}(+ix_{0})$, respectively. Then, the integral is evaluated as

[TABLE]

The derivative is

[TABLE]

By applying inequality (33),

[TABLE]

(the equality holds iff $v=1/3$), the numerator of (34) is bounded below by

[TABLE]

which is positive for $0<v<1/3$. Therefore, $\frac{\mathrm{d}}{\mathrm{d}v}\underline{\mathrm{len}}(v)<0$ for $0<v<1/3$, and $\underline{\mathrm{len}}(v)$ has the unique minimum at $v=1/3$.

Since $s(x;v)\geq\underline{s}(x;v)$,

[TABLE]

Moreover, since $s(x;v)=\underline{s}(x;v)$ at $v=1/3$,

[TABLE]

Therefore,

[TABLE]

Point $v=1/3$ is the unique minimizer, because this is the unique minimizer of $\underline{\mathrm{len}}(v)$.

Figure 2 depicts the objective function $\mathrm{len}(v)$ and its lower bound $\underline{\mathrm{len}}(v)$ for $v\leq 1/3$. ∎

As shown in (19), the information matrix $M_{1/3}$ is the counterpart of the information matrix for the uniform design in the Fourier regression.

Recall the decomposition of $A(a,b,c,d)$ in Proposition 3.8. We already know from Theorem 4.4 that, for $s^{2}+t^{2}=1$,

[TABLE]

Moreover,

[TABLE]

Theorem 4.6 can be written in the following form.

Theorem 4.7**.**

The minimizer of $\mathrm{Vol}_{1}(\gamma_{M^{-1}})$ in (7) over $M\in\mathcal{M}_{P}$ is given when and only when $M$ is of the form:

[TABLE]

Remark 4.8**.**

The minimum tube-volume $M\in\mathcal{M}_{P}$ is attained when and only when the curve

[TABLE]

forms a circle. Moreover, in that case, the circle length is $2\pi\sqrt{2/3}$.

Finally, we characterize the tube-volume optimal design as a three-point design. The polynomial design corresponding to the Fourier uniform design is given in (19). The tube-volume optimal design is obtained as an orbit of the transformation passing through the design in (19). In the following, let

[TABLE]

a three-point uniform design in the Fourier regression.

Theorem 4.9**.**

In the weighted polynomial regression with variance function $\sigma_{P}^{2}(x;a_{0},b_{0},c_{0},\allowbreak d_{0})$, the three-point tube-volume optimal design is $\left\{\begin{matrix}x_{i}\\ p_{i}\end{matrix}\right\}_{i=1,2,3}$, where

[TABLE]

Here, $t^{0}_{i}$ is defined in (35), $a,b,c,d$ are arbitrarily given so that $ad-bc\neq 0$, and $k>0$ is a constant so that $\sum p_{i}=1$. The tube-volume optimal design includes the D-optimal designs as special cases where

[TABLE]

holds for some $s^{2}+t^{2}=1$.

Proof.

The optimal design is the orbit of the Möbius group passing through the design $\left\{\begin{matrix}y_{i}\\ \frac{1}{3}\end{matrix}\right\}_{i=1,2,3}$, $y_{i}=\tan(\pi t^{0}_{i})$, with variance function $\sigma_{P}^{2}(y;1,0,0,1)$. By Theorem 3.17 with $(a^{\prime},b^{\prime},c^{\prime},d^{\prime})=(1,0,0,1)$, we see that $\left\{\begin{matrix}x_{i}\\ \frac{1}{3}\end{matrix}\right\}_{i=1,2,3}$ is an optimal design under variance function $\sigma_{P}^{2}(x;a,b,c,d)$. By Lemma 2.4 with $\sigma_{1}^{2}(x)=\sigma_{P}(x;a,b,c,d)$, $\sigma_{2}^{2}(x)=\sigma_{P}(x;a_{0},b_{0},c_{0},d_{0})$, we have $p_{i}$ in (36). The results for D-optimality is proved in Theorem 3.19. ∎

Theorem 4.10**.**

In the Fourier regression, the three-point tube-volume optimal design is given as

[TABLE]

where $k$ is a normalizing constant so that $\sum_{i}p_{i}=1$, $q\neq 0$, and $r$, $\theta$ are arbitrarily given. In particular, the uniform design (D-optimal design) $\left\{\begin{matrix}t^{0}_{i}-\theta\\ \frac{1}{3}\end{matrix}\right\}_{i=1,2,3}$ is a tube-volume optimal design.

Proof.

Let $x_{i}$ and $p_{i}$ be given in (36) when $(a_{0},b_{0},c_{0},d_{0})=(1,0,0,1)$. Then, $\sigma_{P}^{2}(x;a_{0},b_{0},c_{0},\allowbreak d_{0})=\sigma_{P}^{2}(x)$, and from Theorem 3.3, the optimal design is given by $\left\{\begin{matrix}t_{i}\\ p_{i}\end{matrix}\right\}$, $t_{i}=\frac{1}{\pi}\tan^{-1}(x_{i})$. Let $(a,b,c,d)$ be chosen such that $\varphi^{-1}(\cdot;a,b,c,d)=\varphi(\cdot;q,r,0,1)\circ\varphi(\cdot;s,-t,t,s)$ with $s=\cos(\pi\theta)$, $t=\sin(\pi\theta)$. Then, $\varphi(\tan(\pi t^{0}_{i});s,-t,t,s)=\tan(\pi(t^{0}_{i}-\theta))$ and $x_{i}=q\tan(\pi(t^{0}_{i}-\theta))+r$, and we have $t_{i}=\frac{1}{\pi}\tan^{-1}(x_{i})$ in (37).

On the other hand, since \varphi(\cdot;a,b,c,d)=\varphi(\cdot;s,-t,t,s)\circ\varphi\bigl{(}\cdot;\frac{1}{q},-\frac{r}{q},0,1\bigr{)}, \sigma_{P}^{2}(x_{i};a,b,c,d)\allowbreak=\sigma_{P}^{2}\bigl{(}x_{i};\frac{1}{q},-\frac{q}{r},0,1\bigr{)}=\bigl{(}1+\frac{1}{q^{2}}(x_{i}-r)^{2}\bigr{)}^{2}. Therefore,

[TABLE]

which is equivalent to $p_{i}$ in (37). The uniform design corresponds to the case $(q,r)=(1,0)$. ∎

4.3 Numerical comparisons

Here we conduct a small numerical experiment to see the difference of the width of the simultaneous confidence band under optimal and non-optimal designs. The model we use involves polynomial regression $f(x)=(1,x,x^{2})^{\top}$, $x\in\mathcal{X}=(-\infty,\infty)$, with the variance function of $\sigma^{2}(x)=(1+x^{2})^{2}$. The three point designs,

[TABLE]

and $v=1/12,1/9,1/6,1/4,1/3,1/2$, are compared. The information matrix of $\mathcal{D}(\nu)$ is

[TABLE]

and the length $\mathrm{Vol}_{1}(\gamma_{M(v)^{-1}})$ takes its minimum at $v=1/3$, as proved in Theorem 4.6. Table 1 shows the empirical upper $\alpha$ quantiles $w_{\alpha}$ of the standardized simultaneous confidence bands for the designs $\mathcal{D}(v)$ and their corresponding theoretical values. We generated the random variable

[TABLE]

through simulation with 300,000 replications to obtain the empirical $\alpha$-quantiles $w_{\alpha}$. The corresponding theoretical values by part (i) of Proposition 2.1 are given in parentheses. As the theorems state, the case $v=1/3$ has the narrowest simultaneous confidence band, although the improvement in the width is not substantial.

5 Summary and remaining problems

In this paper, we have proposed the tube-volume (TV) criterion $\mathrm{Vol}_{1}(\gamma_{\Sigma})$ in (7) in experimental design. If a design is tube-volume optimal and simultaneously, D-optimal minimizing $\max_{x\in\mathcal{X}}f(x)^{\top}\Sigma^{-1}f(x)$, the design is optimal that attains the minimum band-width of simultaneous confidence bands.

Then, the proposed criterion was applied to Fourier regression model that is a standard model in linear optimal design theory, and weighted polynomial regression model that is mathematically equivalent to the Fourier regression model. The Möbius group keeps the tube-volume criterion invariant, whereas the subgroup $\mathrm{O}(2,\mathbb{R})$ of the Möbius group keeps the D-criterion invariant.

Using the Möbius invariance, when $n=3$, we found that the tube-volume optimal designs in the Fourier regression and the weighted polynomial regression form an orbit of the Möbius group. The tube-volume optimal designs contain D-optimal designs as special cases. This means that in the Fourier regression, the uniform design is a universal optimal design minimizing both tube-volume criterion and D-criterion.

A conjecture

We conjecture that for all $n$, the tube-volume optimal design is characterized as an orbit of the Möbius group containing D-optimal designs. One supporting observation is that for small $n$ ($n\leq 6$), tube-volume local optimality at the D-optimal designs can be proved by direct calculations. That is, the Hessian matrix of the tube-volume criterion evaluated at the D-optimal design is positive semi-definite, and the null space of the Hessian matrix corresponds to the tangent space of the orbit of Möbius group action. However, the proof for general $n$ remains outstanding. As stated in Remark 4.8, when $n=3$, the trajectory of the normalized regression basis of the minimum length is a circle. Although the trajectory cannot be a circle for $n\geq 4$, some idea of the isoperimetric inequality may be useful.

Multivariate extension

Throughout the paper, we just dealt with the case where the explanatory variable is one-dimensional. However, the volume-of-tube method works for the construction of the simultaneous confidence bands for regression with multidimensional explanatory variables except for the conservativeness (ii) of Proposition 2.1, and the volume-optimality is well-defined. For example, we can discuss the volume-optimality of the $p$-variate polynomial regression model with the basis vector

[TABLE]

By the same argument as the univariate case, we can prove that the multivariate Möbius transform $\varphi:\overline{\mathbb{R}}^{p}\to\overline{\mathbb{R}}^{p}$ defined by

[TABLE]

where $A\in\mathbb{R}^{p\times p}$, $b,\,c\in\mathbb{R}^{p\times 1}$, $d\in\mathbb{R}$ (e.g., Kato and McCullagh (2014)) remains the invariance (volume preserving property) $\mathrm{Vol}_{p}(\gamma_{M^{-1}})=\mathrm{Vol}_{p}(\gamma_{(AMA^{\top})^{-1}})$ of Theorem 3.16. However, the treatment of the multidimensional case (see, e.g., Lasserre (2009) for the moment cone) remains a future topic of research.

Application to other regression models

The application of the TV-criterion to other regression models remains an important research topic. As a simple example, consider the Fourier and the weighted polynomial regressions (8) and (9) whose explanatory domain is a finite interval $\mathcal{X}=[A,B]$. Then, using arguments similar to those used in Section 4, we can show that the TV-optimal design is an improper two-point design with masses at $A$ and $B$. This does not coincide with the D-optimal design.

From this observation, we pose two problems: (i) To characterize the models in which the proper TV-optimal design exists, and the TV-optimal and D-optimal designs are compatible. (ii) How to combine the TV-optimal and D-optimal designs when they are different, for example, a mixture of the D- and TV-optimal designs with appropriate weights.

Appendix A Appendix: Proofs

A.1 Proof of Proposition 2.2

Proof.

Let $\eta=\xi/\|\xi\|$, where $\xi=\Sigma^{-\frac{1}{2}}(\widehat{b}-b)$. $\eta$ is uniformly distributed on the unit sphere $\mathbb{S}^{n-1}$. Let $\psi(x)=\Sigma^{\frac{1}{2}}f(x)/\|\Sigma^{\frac{1}{2}}f(x)\|$. Then, under the assumption that any connected component of $\gamma_{\Sigma}$ is not a closed curve, the upper tail probability of $\max_{x\in\mathcal{X}}|\eta^{\top}\psi(x)|$ is bounded above by the expectation of the Euler-characteristic $\chi(A_{c})$ of the excursion set:

[TABLE]

That is, by Proposition 3.2 and (3.10) of Takemura and Kuriki (2002),

[TABLE]

where $B_{\frac{k}{2},\frac{n-k}{2}}$ is a random variable distributed as the beta distribution with parameter \bigl{(}\frac{k}{2},\frac{n-k}{2}\bigr{)}.

Let $\widetilde{\xi}$ be a copy of $\xi$ distributed independently of $\xi$ and $B_{\frac{k}{2},\frac{n-k}{2}}$. Then, we see the equivalence in distribution:

[TABLE]

(38) can be proved by checking the Mellin transforms. By letting $c:=c/\|\widetilde{\xi}\|$, and taking the expectations with respect to $\widetilde{\xi}$, we have (6). ∎

A.2 Proof of Lemma 3.4

Proof.

Let $t_{k}=k/n-(n+1)/(2n)$ and $x_{k}=\tan(\pi t_{k})$. The $(i,j)$ element of $M$ is

[TABLE]

Then, apply Lemma A.1 below. ∎

Lemma A.1**.**

Let $t_{i}=i/n-d$, $i=1,\ldots,n$, where $d$ is a constant. Then,

[TABLE]

Proof.

From the duplication formula for the gamma function, it suffices to show that

[TABLE]

Let $\omega=e^{i\pi/n}$, $c=e^{-i\pi d}$. The left-hand side times $(2i)^{2k}2^{2n-2k-2}=2^{2n-2}(-1)^{k}$ is

[TABLE]

Here, the contribution of the summation with respect to $j$ is

[TABLE]

Hence, (39) is equal to

[TABLE]

The last equality follows from (1.41) of Gould (2010). ∎

A.3 Proof of Lemma 4.2

The proof of Lemma 4.2 is divided into three parts (lemmas).

Lemma A.2**.**

For $M\in\mathcal{M}_{P}$, there exist $u,w>1$ such that

[TABLE]

Lemma A.3**.**

For $u,w>1$, there exist $\tilde{u},\tilde{v},\tilde{w}>0$, $\tilde{v}<\sqrt{\tilde{u}\tilde{w}}$ such that

[TABLE]

Lemma A.4**.**

For $\tilde{u},\tilde{v},\tilde{w}>0$, $\tilde{v}<\sqrt{\tilde{u}\tilde{w}}$, there exist $v\in(0,1)$ such that

[TABLE]

Proof of Lemma A.2.

We start from a canonical form in (29):

[TABLE]

where $f_{P}(x)=(1,x,x^{2})^{\top}$, $f_{P}(\pm\infty)=(0,0,1)^{\top}$. For $A=A(a,b,c,d)$ with $a=w_{2}^{1/4}(x_{1}-x_{2})$, $b=-w_{2}^{1/4}x_{1}$, $c=0$, $d=-w_{2}^{1/4}$, we have

[TABLE]

∎

Proof of Lemma A.3.

Let

[TABLE]

We confirm that equation $AMA^{\top}=\widetilde{M}$ has a solution $(a,b,c,d)$ such that $ad-bc\neq 0$. It is enough to show that under the assumption $a,d\neq 0$, a solution $(a,b,c,d)$ satisfies

[TABLE]

Solving $(AMA^{\top})_{1,2}=0$ with respect to $b$ yields

[TABLE]

where

[TABLE]

if $f_{1}(c/d;u)\neq 0$. Substituting (41) into $(AMA^{\top})_{2,3}$, we have

[TABLE]

where

[TABLE]

Substituting (41) into $ad-bc$, we have

[TABLE]

In the numerator of (42), if $(v-1)c^{4}+(u-1)d^{4}+(c+d)^{4}=0$, then $c=d=0$, and hence, $ad-bc=0$.

Now, we examine whether for all $u,v>1$, there is a real $x$ such that

[TABLE]

Once a solution $x=x^{*}$ is obtained, we have a solution that $a,d\neq 0$ are given arbitrarily, $c=dx^{*}$, and $b$ is determined from $(a,c,d)$ in (41).

Write $f(\cdot)=f(\cdot;u,v)$ shortly. It is easily shown that when $u,v>1$, neither

[TABLE]

nor

[TABLE]

is true, where $f(\pm\infty)=\lim_{c\to\pm\infty}c^{-6}f(c)$. This means that $f(x;u,v)=0$ has a real solution $x=x^{*}$.

In order to check $f_{1}(x^{*};u)\neq 0$ and $f_{2}(x^{*};u,v)\neq 0$, we need to check whether $f$ and $f_{1}$ (or $f_{2}$) have a common factor. For this purpose, we calculate the resultants $R(f,f_{1})$ and $R(f,f_{2})$:

[TABLE]

where

[TABLE]

(For resultant, see, e.g., Prasolov (2004). Applications in statistics can be found in Drton, et al. (2009).) As shown in Lemma A.5 later, in the region $u,v>1$, $h(u,n)=0$ iff $u=2$. Therefore, when $u\neq 2$, we have established that $x=x^{*}$ satisfying (43) exists, and hence, a solution $(a,b,c,d)$ satisfying (40) exists.

When $u=2$, (42) is reduced to

[TABLE]

and $c=0$ is a solution. From (41), $b=-a/2$, and $a\neq 0$, $d\neq 0$ are arbitrarily given. In fact, for $A=A(a,-a/2,0,d)$,

[TABLE]

∎

Lemma A.5**.**

Let $h(u,v)$ be defined in (44). When $u,v>1$, $h(u,v)\geq 0$, and the equality holds iff $u=v=2$.

Proof.

Fix $u$ and consider $h(u,v)$ as a function of $v$. Note that $h(u,1)=(u-1)^{3}>0$.

[TABLE]

$h_{v}(u,1)=3u(u-1)^{2}>0$. It is easy to see that $h_{v}(u,v)=0$ has a real solution iff $3/2\leq u\leq 3$. Therefore, when $u\leq 3/2$ or $u\geq 3$, $h_{v}(u,v)$ is always positive. Combined with $h(u,1)>0$, this means that $h(u,v)>0$.

Consider the case $3/2<u<3$. The largest zero of $h_{v}(u,v)$ is

[TABLE]

At this point, the function $h(u,v)$ takes a local minimum

[TABLE]

It is easy to check that $-54+108u-72u^{2}+18u^{3}-u^{4}>0$ for $3/2<u<3$. Hence, $h(u,v_{*}(u))\geq 0$ for $3/2<u<3$ and the equality holds when $u=2$. When $u=2$, $v_{*}(2)=2$. ∎

Proof of Lemma A.4.

For $A=A(a,b,c,d)$ with $b=c=0$,

[TABLE]

By letting $a=\tilde{w}^{-1/4}$, $d=\tilde{u}^{-1/4}$, $v=a^{2}d^{2}\tilde{v}=\tilde{v}/\sqrt{\tilde{u}\tilde{w}}$, we obtain the result. ∎

Remark A.6**.**

In the proof of Lemma A.4, by letting $d=\tilde{u}^{-1/4}$, $a=\tilde{v}^{-1/2}/\sqrt{3}d=\tilde{u}^{1/4}/\sqrt{3}\tilde{v}^{1/2}$, $w=a^{4}\tilde{w}=\tilde{u}\tilde{w}/9\tilde{v}^{2}$, we have another representative group element

[TABLE]

A.4 Proof of Lemma 4.3

Proof.

The $(1,1)$, $(2,1)$, $(3,2)$, $(3,3)$ components of the equation $M_{v^{\prime}}=AM_{v}A^{\top}$ are

[TABLE]

respectively. By solving $(AM_{v}A^{\top})_{2,1}=0$, we have

[TABLE]

when $d\neq 0$. ($c$ and $d$ cannot be 0 simultaneously because $ad-bc\neq 0$.) Suppose first that $d\neq 0$. Substituting this into $(AM_{v}A^{\top})_{3,2}$, we have

[TABLE]

This becomes 0 when (i) $c=0$, (ii) $c=\pm d$, or (iii) $v=1/3$. We try to solve the equation $(AM_{v}A^{\top})_{1,1}=(AM_{v}A^{\top})_{3,3}=1$ in each case. Note that $a=0$ cannot be a solution, because $b$ becomes 0 and $ad-bc=0$.

(i) If $c=0$, then $b$ becomes 0 and $(AM_{v}A^{\top})_{1,1}=d^{4}$, $(AM_{v}A^{\top})_{3,3}=a^{4}$. Hence, $(a,b,c,d)=(\pm 1,0,0,\pm 1)$ (four ways) are the solutions. In each case,

[TABLE]

(ii) If $c=\pm d$, then $b=\mp a$, $(AM_{v}A^{\top})_{1,1}=2c^{4}(1+3v)=1=(AM_{v}A^{\top})_{3,3}=2a^{4}(1+3v)$, and hence, $c=\pm a$. Therefore,

[TABLE]

are solutions. In each case,

[TABLE]

(iii) When $v=1/3$ and $d\neq 0$, we have $b=-ac/d$ and

[TABLE]

Hence, $b=\pm\sqrt{1-a^{2}}$. When $d=a$, $c=-b=\mp\sqrt{1-a^{2}}$. When $d=-a$, $c=b=\pm\sqrt{1-a^{2}}$. In summary,

[TABLE]

In each case,

[TABLE]

(iii’) When $v=1/3$ and $d=0$, $(AM_{v}A^{\top})_{2,1}=ac^{3}=0$. Because $ad-bc\neq 0$, $c\neq 0$, and $a=0$. In this case, $(AM_{v}A^{\top})_{3,2}=0$, $(AM_{v}A^{\top})_{1,1}=c^{4}=1$, $(AM_{v}A^{\top})_{3,3}=b^{4}=1$. Hence, $(a,b,c,d)=(0,\pm 1,\pm 1,0)$ (four ways) are the solutions. In each case,

[TABLE]

∎

Acknowledgments

The authors are grateful to Yasuhiro Omori and Tatsuya Kubokawa for their helpful comments. This work was supported by JSPS KAKENHI Grant Number 16H02792.