Asymptotic comparison of two-stage selection procedures under quasi-Bayesian framework

TL;DR

This paper analyzes two-stage selection procedures for Gaussian populations, confirming a conjecture about their asymptotic efficiency using a quasi-Bayesian framework and exploring open questions on Student-t maxima.

Contribution

It introduces a quasi-Bayesian model validating a conjecture on the asymptotic efficiency of selection procedures and discusses open problems on Student-t distribution maxima.

Findings

Conjecture on asymptotic efficiency ratio is validated under the quasi-Bayesian model.

Provides insights into the extreme value distribution of Student-t maxima.

Highlights open questions in the theory of Student-t maxima.

Abstract

This paper revisits the procedures suggested by Dudewicz and Dalal (1975) and Rinott (1978) which are designed for selecting the population with the highest mean among independent Gaussian populations with unknown and possibly different variances. In a previous paper Jacobovic and Zuk (2017) made a conjecture that the relative asymptotic efficiency of these procedures equals to the ratio of two certain sequences. This work suggests a quasi-Bayesian modelling of the problem under which this conjecture is valid. In addition, this paper motivates an open question regarding the extreme value distribution of the maxima of triangular array of independent student-t random variables with an increasing number of degrees of freedom.

Equations

$$\Theta (\Delta ):=\left\{\tilde{\theta }\in {\mathbb{R}}^{k+1};\underset{1\le i_1<i_2\le k+1}{\min }|{\tilde{\theta }}_{i_1}-{\tilde{\theta }}_{i_2}|>\Delta \right\}.$$

$$N_{i}^{l}(k)=\max\bigg{\{}N_{0}(k)+1,\bigg{\lceil}\bigg{(}\frac{h_{k}^{l}}{\Delta}\bigg{)}^{2}S_{i}^{2}(k)\bigg{\rceil}\bigg{\}}.$$

$$p={\int }_{-\infty }^{\infty }{G}_{{\nu }_k}^k(t+h)g_{{\nu }_k}(t)dt$$

$$p={\left[{\int }_{-\infty }^{\infty }{G}_{{\nu }_k}(t+h)g_{{\nu }_k}(t)dt\right]}^k.$$

$$\eta :=\underset{k\to \infty }{\lim }\frac{{\sum }_{i=1}^{k+1}E{N}_i^2(k)}{{\sum }_{i=1}^{k+1}E{N}_i^1(k)}$$

$$\sum _{i=1}^{k+1}E{N}_i^2(k)\sim (k+1){\left(\frac{h_k^j\sigma }{\Delta }\right)}^2\text{as}k\to \infty$$

$${\alpha }_k^l:$$

$$=\frac{1}{(h_k^l/\Delta {)}^2}\mathbb{E}{N}_1^l(k)$$

$$=\mathbb{E}\max \left\{\frac{N_0+1}{(h_k^l/\Delta {)}^2},\frac{\lceil S_1^2{\left(\frac{h_k^l}{\Delta }\right)}^2\rceil }{(h_k^l/\Delta {)}^2}\right\}.$$

$$\mathbb{E}{S}_1^2={\mathbb{E}}_{\Sigma }\mathbb{E}\left(S_1^2|\Sigma \right)=\mathbb{E}{\sigma }_1^2={\sigma }^2<\infty$$

$$Y:=\frac{N_0+1}{(h_{k_0^l}^j/\Delta {)}^2}+S_1^2+{\left(\frac{h_{k_0^l}^j}{\Delta }\right)}^{-2}$$

$$\exists \underset{k\to \infty }{\lim }{\alpha }_k^l$$

$$\stackrel{DCT}{=}\mathbb{E}\left[\underset{k\to \infty }{\lim }\max \left\{\frac{N_0+1}{(h_k^l/\Delta {)}^2},\frac{\lceil S_1^2{\left(\frac{h_k^l}{\Delta }\right)}^2\rceil }{(h_k^l/\Delta {)}^2}\right\}\right]$$

$$=\mathbb{E}{S}_1^2={\sigma }^2.$$

$$\eta =\underset{k\to \infty }{\lim }\frac{h_k^2}{h_k^1}=2^{2/\nu }$$

$$\mathbb{E}{N}_l(k)\sim (k+1){\left(\frac{h_k^l}{\Delta }\right)}^2\mathbb{E}{Y}_l\text{as}k\to \infty$$

$${\alpha }_k^l:=\frac{{\sum }_{i=1}^{k+1}\mathbb{E}{N}_i^l(k)}{(k+1)(h_k^l/\Delta {)}^2}$$

$$\frac{N_0^l(k)-1}{{\sigma }_1^2}{S}_1^2(k)\sim {\chi }_{(N_0^l(k)-1)}^2.$$

$$Var\left[S_{1}^{2}(k)\big{|}\Sigma\right]=\left[\frac{\sigma_{1}^{2}}{N_{0}^{l}(k)-1}\right]^{2}2\left[N_{0}^{l}(k)-1\right]\leq 2\left(\sigma_{1}^{2}\right)^{2}<\infty\,.$$

$$\displaystyle\sup_{k\in\mathbb{N}}\mathbb{E}\left\{\left[\frac{N_{0}^{l}(k)+1}{(h_{k}^{l}/\Delta)^{2}}+S_{1}^{2}(k)+\left(\frac{\Delta}{h_{k}^{l}}\right)^{2}\right]^{2}\bigg{|}\Sigma\right\}<\infty$$

$$\lim_{k\to\infty}\mathbb{E}\left(M_{k}|\Sigma\right)=\mathbb{E}\left(\lim_{k\rightarrow\infty}M_{k}|\Sigma\right)=\mathbb{E}\left(\max\left\{L_{l},\sigma_{1}^{2}\right\}\big{|}\Sigma\right)=\max\left\{L_{l},\sigma_{1}^{2}\right\}\,.$$

$$\underset{k\in \mathbb{N}}{\sup }{\mathbb{E}}_{\Sigma }{\mathbb{E}}^2\left(M_k|\Sigma \right)\le \underset{k\in \mathbb{N}}{\sup }{\mathbb{E}}_{\Sigma }{\left[\frac{N_0^l(k)+1}{(h_k^l/\Delta {)}^2}+{\sigma }_1^2+{\left(\frac{\Delta }{h_k^l}\right)}^2\right]}^2<\infty$$

$$\underset{k\to \infty }{\lim }{\alpha }_k^l=\underset{k\to \infty }{\lim }{\mathbb{E}}_{\Sigma }\mathbb{E}(M_k|\Sigma )={\mathbb{E}}_{\Sigma }\underset{k\to \infty }{\lim }\mathbb{E}(M_k|\Sigma )=\mathbb{E}\max \left\{L,{\sigma }_1^2\right\}$$

$$\exists \underset{k\to \infty }{\lim }\frac{h_k^2}{h_k^1}=:H,$$

Full text

1 Introduction

Consider the problem of a decision maker who has an access to noisy observations taken from a set of populations and has to select the best one. In general, the notion of the best population may be determined with respect to different criteria including highest mean, lowest variance, highest $R$-squared with respect to some target variable and etc. The branch of decision theory which deals with this kind of decisions is known as selection procedures or multiple decision procedures. For books regarding this subject see e.g [6, 7]. Traditionally, these problems were investigated with small number of populations. Some modern applications of this theory involve gene-expression datasets and discrete event simulation which is a popular methodology for finding the optimal (or near-optimal) system design (e.g. populations). A comprehensive survey of these applications is provided by [10]. The point is that both of these modern applications usually involve enormous number of populations. Respectively, this paper is devoted for analysing the asymptotic performances of two selection procedures which were suggested respectively by Dudewicz and Dalal [4] and Rinott [11]. These procedures were designed in order to find the population with the highest mean among a set of independent Gaussian populations with possibly different variances which are unknown to the user. What we do here is to define their relative asymptotic efficiency as the number of populations tends to infinity. Then, this work suggests a quasi-Bayesian model under which the conjecture of [10] stating that the asymptotic relative efficiency equals to the limit of a ratio of two sequences $(h_{k}^{1})$ and $(h_{k}^{2})$ to be later explained is valid. The rest is organized as follows: Section 2 provides the model description with further details about the above-mentioned procedures. Section 3 includes the main results and proofs. Finally, Section 4 contains a discussion regarding the way that this work motivates an open question about the asymptotic distribution of a sequence of maxima generated by a triangular array of independent student-t random variables (r.v’s) with an increasing number of degrees of freedom (d.f’s).

2 Model description

Let $\Delta>0$ and consider a sequence of random variables $\Sigma=\{\sigma_{i}^{2}\}_{i=1}^{\infty}$ which are positive with probability one. For each $k\in\mathbb{N}$ the following is a quasi-Bayesian model of samplings drawn from $k+1$ Gaussian populations. We denote these populations by $\Pi^{(k)}_{1},\ldots,\Pi^{(k)}_{k+1}$ and let $X_{j}^{i}(k)$ be the $j$th sampling from the population $\Pi^{(k)}_{i}$. Now, consider a scalar vector $\theta^{(k)}=(\theta_{1}^{(k)},\ldots,\theta^{(k)}_{k+1})$ which belongs to

[TABLE]

The r.v’s $X_{j}^{i}(k);i=1,\ldots,k+1\ ,\ j\in\mathbb{N}$ are such that given $\Sigma$, they are independent and satisfying $X_{j}^{i}(k)\sim N(\theta_{i}^{(k)},\sigma_{i}^{2}),\forall i,j\in\mathbb{N}$.

2.1 Selection problem

It is assumed that $\theta^{(1)},\theta^{(2)},\ldots$ are unknown vectors. More assumptions are that all components of $\Sigma$ are unobservable while $\Delta$ is a parameter which is set by the user with respect to her preferences. In particular, commonly, $\Delta$ is considered as a parameter which reflects the indifference level of the user regarding two populations with distinct means (see e.g. [2]). With respect to this setup, fix $k\in\mathbb{N}$ and the purpose is to pinpoint the population with the highest mean among the populations $\Pi^{(k)}_{1},\ldots,\Pi^{(k)}_{k+1}$. To this end, a selection procedure, i.e. a sampling policy with a selection rule to pinpoint the correct population is required. The requirement from such a rule is to identify the correct population (PCS) with probability which is not less than $p\in(0,1)$ where $p$ is a confidence parameter to be determined by the user.

2.2 Two-stage selection procedures

Two optional two-stage procedures were suggested respectively by Dudewicz and Dalal [4] (procedure 1) and Rinott [11] (procedure 2). Roughly speaking, the first stage is about drawing constant number of samplings $N_{0}(k)$ from every population. This sample is used to estimate the variance of each population. We denote the estimated variances by $S_{1}^{2}(k),\ldots,S_{k+1}^{2}(k)$. Then, the second stage tells the user to draw more samplings from each population. Importantly, for each population, the additional sample size taken at the second stage is an increasing function of the corresponding empirical variance. Finally, for each population, the user should average the samplings with respect to some choice of weights and pick the population which is associated with the highest weighted-average. The exact details of these procedures are summarized together at [10]. For our purposes, we only recall the relevant details. To start with, let $G_{\nu_{k}}(\cdot)$ and $g_{\nu_{k}}(\cdot)$ be the c.d.f. and p.d.f of student’s-t distribution with $\nu_{k}:=N_{0}(k)-1$ degrees of freedom (d.f’s). In addition, for each $i$, let $N_{i}^{1}(k),N_{i}^{2}(k)$ be the number of samplings taken from the $i$th population by the procedures 1 and 2. It is known that for each $l=1,2$

[TABLE]

where $h_{k}^{1}$ and $h_{k}^{2}$ are defined respectively as the unique solutions of the following equations in $h$

[TABLE]

2.3 Relative asymptotic efficiency

To evaluate the performance of the procedures which were introduced earlier, it is possible to consider the expected number of samplings that each of them requires. It is straightforward that when the number of populations is taken to infinity, then the corresponding performance measure also tends to infinity. Therefore, in order to compare the asymptotic performance of these procedures, we are looking at the relative asymptotic efficiency which is defined as the next limit whenever it exists.

Definition 1

The relative asymptotic efficiency of the above-mentioned procedures is given by the limit

[TABLE]

if exists. Otherwise, it is not defined.

3 Main results

The next theorem refers to the case where the sample-size of the first stage is constant with respect to $k$.

Theorem 1

Let $N_{0}(k)=N_{0}=\nu+1,\forall k\in\mathbb{N}$. If

$\sigma_{i}^{2}\stackrel{{\scriptstyle d}}{{=}}\sigma_{j}^{2},\forall i,j$** 2. 2.

$\sigma^{2}:=\mathbb{E}\sigma_{1}^{2}<\infty$**

then for every $l=1,2$

[TABLE]

and hence $\eta=\lim_{k\to\infty}\frac{h_{k}^{2}}{h_{k}^{1}}=2^{2/\nu}$ .

Proof: To start with, observe that $\sigma_{1}^{2},\sigma^{2}_{2},\ldots$ are equally distributed and that for each $i=1,\ldots,k+1$ the distribution of $S_{i}^{2}(k)$ given $\Sigma$ is determined uniquely by $\sigma_{i}^{2}$. Especially, since $N_{0}(k)=N_{0},\forall k\in\mathbb{N}$, then this distribution doesn’t depend on $k$. Therefore, we shall write that for each $i$, $S^{2}_{i}(k)\sim S_{1}^{2}$. Now, fix $l\in\{1,2\}$ and notice that $\{h_{k}^{l}\}$ is a monotonic sequence such that $h_{k}^{l}\to\infty$ as $k\to\infty$. Therefore, we may let $k^{l}_{0}:=\min\{k;h_{k}^{l}>0\}$ and fix $k>k_{0}^{l}$. Then, by their definitions, the r.v’s $N_{1}^{l}(k),\ldots,N^{l}_{k+1}(k)$ are equally distributed. This can be used in order to show that

[TABLE]

Now, recall that given $\Sigma$, $S_{1}^{2}$ is an unbiased estimator for $\sigma^{2}_{1}$. Thus, the law of total expectation implies that

[TABLE]

which means that $S_{1}^{2}$ is an integrable r.v. In addition, since $\{h_{k}^{l}\}$ is a non-decreasing sequence which is positive from $k_{0}^{l}$, then it can be seen that the maximum inside the expectation is non-negative and bounded by

[TABLE]

which is an integrable r.v. Thus, the dominated convergence theorem (DCT) may be applied in order to derive the limit

[TABLE]

In particular, notice that the derivation of the limit inside the expectation was made by using the facts that $h_{k}^{l}\to\infty$ as $k\to\infty$ and $S_{1}^{2}>0$ while $N_{0}$ is a constant. This establishes the first order approximation for $\sum_{i=1}^{k+1}\mathbb{E}N_{i}^{l}(k)$ as $k\to\infty$. Therefore, it is an immediate result that

[TABLE]

where the last equality holds due to Theorems 4.1 and 4.2 of [10].    

The next theorem refers to the case where the initial sample size tends to infinity as the number of populations tends to infinity.

Theorem 2

Assume that

$\sigma_{i}^{2}\stackrel{{\scriptstyle d}}{{=}}\sigma_{j}^{2},\forall i,j$** 2. 2.

$\mathbb{E}\left[(\sigma_{1}^{2})^{2}\right]<\infty$** 3. 3.

$2\leq N_{0}(k)\to\infty$* as $k\to\infty$* 4. 4.

$\exists\lim_{k\to\infty}\frac{N_{0}(k)}{(h_{k}^{l}/\Delta)^{2}}=:L_{l}<\infty\ \ ,\ \ \forall l=1,2$**

and let $Y_{l}=\max\{L_{l},\sigma_{1}^{2}\},\forall l=1,2$. Then, for every $l=1,2$

[TABLE]

and $\eta=\lim_{k\to\infty}\frac{h_{k}^{2}}{h_{k}^{1}}$ if such limit exists.

Proof: Let $l=1,2$. Using the same arguments appeared in the proof of Theorem 1, for every $k\in\mathbb{N}$

[TABLE]

Denote the maximum inside the above-mentioned expectation by $M_{k}$ and recall that given $\Sigma$, $S_{1}^{2}(k)$ is an unbiased estimator of $\sigma_{1}^{2}$. Thus, it is known that given $\Sigma$

[TABLE]

Therefore, Equation (1) clearly implies that

[TABLE]

Consequently, since both the first and second moments of $S_{1}^{2}(k)$ given $\Sigma$ are finite and constants with respect to $k$, then it may be deduced that $\sup_{k\in\mathbb{N}}\mathbb{E}(M_{k}^{2}|\Sigma)$ is bounded by

[TABLE]

where we have used the facts that $h_{k}^{l}\to\infty$ as $k\to\infty$ and $L_{l}<\infty$ in order to bound the first and third summands inside the expectation. Thus, by the theorem of de la Vallée Poussin with test function $g(x)=x^{2},\forall x\in\mathbb{R}_{+}$, deduce that given $\Sigma$, $\{M_{k};k\in\mathbb{N}\}$ is a sequence of r.v’s which are uniformly integrable. Thus, since $N_{0}(k)\to\infty$ as $k\to\infty$, the strong law of large numbers implies that given $\Sigma$, $S_{1}(k)\xrightarrow{1}\sigma^{2}_{1}$ as $k\to\infty$. Therefore, Vitali’s convergence theorem implies that

[TABLE]

In addition, observe that

[TABLE]

where we have used the detail that $\sigma_{1}^{2}$ is associated with finite second moment along with the fact that for every $k\in\mathbb{N}$, $S_{1}^{2}(k)$ is an unbiased estimator of the variance, i.e. $\mathbb{E}\left[S_{1}^{2}(k)|\Sigma\right]=\sigma_{1}^{2},\forall k\in\mathbb{N}$. Thus, once again, by the theorem of de la Vallée Poussin with test function $g(x)=x^{2},\forall x\in\mathbb{R}_{+}$, deduce that $\left\{\mathbb{E}(M_{k}|\Sigma);k\in\mathbb{N}\right\}$ is a sequence of r.v’s which are uniformly integrable. Therefore, Vitali’s convergence theorem implies that

[TABLE]

where the last equality holds due to Equation (2). Finally, if

[TABLE]

then we shall deduce that $\eta=H$. Especially observe that $EY<\infty$ stems directly from the Theorem assumptions.

4 Discussion

Theorem 1 establishes the result that for a fixed initial sample size, with the current quasi-Bayesian assumptions, the asymptotic performance of procedure suggested by Dudewicz and Dalal is better than the performance of the procedure suggested by Rinott. While for this case, we have made strict conclusions, for the other case where the initial sample size tends to infinity as the number of populations tends to infinity things are different. Theorem 2 establishes a connection between the asymptotic performance of the procedures to the sequences $(h_{k}^{1})$ and $(h_{k}^{l})$. However, it is still not clear for which sequences $\left(N_{0}(k)\right)$, condition 4 of Theorem 2 is satisfied? Intuitively speaking, it seems reasonable that when $N_{0}(k)$ tends to infinity slow enough, then this condition holds, but still this requires further investigation. In addition, for this case, it is not clear, when it exists, what is the limit of $h_{k}^{2}/h_{k}^{1}$ as $k\to\infty$? To solve this questions, it seems that the same methods used by [10] in order to derive first order approximations of $(h_{k}^{1})$ and $(h_{k}^{2})$ for the fixed initial sample size should work here as well. However, the main obstacle in that way is the need to specify the extreme value distribution of some triangular arrays of random variables. In the case of $h_{k}^{1}$, for each $k$, it is necessary to find the limit distribution of the maximum of $k$ i.i.d student-t r.v’s with $\nu_{k}=N_{0}(k)-1$ d.f’s. Similarly, for the case of $h_{k}^{2}$ we should derive the limit distribution of $k$ i.i.d r.v’s such that each of them is distributed like a sum of two independent student-t r.v’s with $\nu_{k}$ d.f’s. To the best of our knowledge non of these triangular arrays have references in literature. For existing literature about extreme value distributions of triangular arrays see e.g. [1, 3, 5, 8, 9].