Uniformly consistent proportion estimation for composite hypotheses via integral equations: "the case of Gamma random variables"

TL;DR

This paper develops uniformly consistent estimators for the proportion of false null hypotheses in Gamma-distributed variables under composite nulls, using integral equations and harmonic analysis, without relying on p-values.

Contribution

It introduces novel estimators based on integral equations for composite hypotheses involving Gamma distributions, expanding the tools for multiple testing.

Findings

Constructed estimators are uniformly consistent for Gamma distributions.

Provided estimators for functions of bounded variation on the null.

Applicable to high-dimensional models and mixture scenarios.

Abstract

We consider estimating the proportion of random variables for two types of composite null hypotheses: (i) the means of the random variables belonging to a non-empty, bounded interval; (ii) the means of the random variables belonging to an unbounded interval that is not the whole real line. For each type of composite null hypotheses, uniformly consistent estimators of the proportion of false null hypotheses are constructed for random variables whose distributions are members of the Gamma family. Further, uniformly consistent estimators of certain functions of a bounded null on the means are provided for the random variables mentioned earlier. These functions are continuous and of bounded variation. The estimators are constructed via solutions to Lebesgue-Stieltjes integral equations and harmonic analysis, do not rely on a concept of p-value, and have various applications.ce via mixture models, and may be used to estimate the sparsity level in high-dimensional Gaussian linear models.

Equations

$${\pi }_{1,m}=m^{-1}\left|\left\{i\in \left\{1,...,m\right\}:{\mu }_i\in {\Theta }_1\right\}\right|$$

$$\psi \left(t,\mu \right)=\int K\left(t,x\right)d{F}_{\mu }\left(x\right),\forall \mu \in U.$$

$${\phi }_m\left(t,\mathbf{\mu }\right)=m^{-1}\sum _{i=1}^m\left\{1-\psi \left(t,{\mu }_i\right)\right\}$$

$${\hat{\phi }}_m\left(t,\mathbf{z}\right)=m^{-1}\sum _{i=1}^m\left\{1-K\left(t,z_i\right)\right\}$$

$$e_m\left(t\right)={\hat{\phi }}_m\left(t,\mathbf{z}\right)-{\phi }_m\left(t,\mathbf{\mu }\right)$$

$$\mathrm{Pr}\left\{\left|{\pi }_{1,m}^{-1}{\hat{\phi }}_m\left(t_m,\mathbf{z}\right)-1\right|\to 0\right\}\to 1\text{as }m\to \infty .$$

$$\frac{d\beta }{d\nu }\left(x\right)=x^{\sigma -1}{e}^{-x}{\left\{\Gamma \left(\sigma \right)\right\}}^{-1}{1}_{\left(0,\infty \right)}\left(x\right),$$

$$f_{\theta }\left(x\right)={\left\{\Gamma \left(\sigma \right)\right\}}^{-1}{\left(1-\theta \right)}^{\sigma }{e}^{-\left(1-\theta \right)x}{x}^{\sigma -1}{1}_{\left(0,\infty \right)}\left(x\right),$$

$${\tilde{f}}_{\tilde{\theta }}\left(x\right)={\left\{\Gamma \left(\sigma \right)\right\}}^{-1}{\tilde{\theta }}^{\sigma }{e}^{-\tilde{\theta }x}{x}^{\sigma -1}{1}_{\left(0,\infty \right)}\left(x\right)$$

$${\tilde{c}}_n\left(\theta \right)=\int x^n{G}_{\theta }\left(dx\right)=\Gamma \left(n+\sigma \right){\left(1-\theta \right)}^{-n}{\left\{\Gamma \left(\sigma \right)\right\}}^{-1}\text{for}n=0,1,2,\cdots ,$$

$$\psi \left(t,\theta \right)=\int K\left(t,x\right)d{G}_{\theta }\left(x\right)\text{ for }{G}_{\theta }\in \mathcal{F}.$$

$${\pi }_{1,m}=m^{-1}\left|\left\{i\in \left\{1,...,m\right\}:{\mu }_i\in {\Theta }_1\right\}\right|$$

$$\left\{\begin{array}[c]{l}K_{1,0}\left(t,x;\mu^{\prime}\right)={\int_{\left[-1,1\right]}}\dfrac{\omega\left(s\right)\cos\left\{ts\left(x-\mu^{\prime}\right)\right\}}{r_{\mu^{\prime}}\left(ts\right)}ds\\ K_{3,0}\left(t,x;\theta^{\prime}\right)={\int_{\left[-1,1\right]}}\sum_{n=0}^{\infty}\left(\tilde{a}_{n}n!\right)^{-1}\left(-tsx\right)^{n}\cos\left\{2^{-1}\pi n+ts\xi\left(\theta^{\prime}\right)\right\}\omega\left(s\right)ds\end{array}\right.$$

$$\left\{\begin{array}[c]{l}\psi_{1,0}\left(t,\mu;\mu^{\prime}\right)=\int K_{1,0}\left(t,x;\mu^{\prime}\right)dF_{\mu}\left(x\right)\text{ if }\mathcal{F}\text{ is a Type I location-shift family}\\ \psi_{3,0}\left(t,\theta;\theta^{\prime}\right)=\int K_{3,0}\left(t,x;\theta^{\prime}\right)dG_{\theta}\left(x\right)\text{ if }\mathcal{F}\text{ is the Gamma family}\end{array}\right..$$

$$\left\{\begin{array}[c]{l}\psi_{1,0}\left(t,\mu;\mu^{\prime}\right)={\int_{\left[-1,1\right]}}\omega\left(s\right)\cos\left\{ts\left(\mu-\mu^{\prime}\right)\right\}ds,\\ \psi_{3,0}\left(t,\theta;\theta^{\prime}\right)={\int_{\left[-1,1\right]}}\cos\left[ts\left\{\xi\left(\theta^{\prime}\right)-\xi\left(\theta\right)\right\}\right]\omega\left(s\right)ds.\end{array}\right.$$

$${\psi }^{†}\left(t,\mu \right)=\int K^{†}\left(t,x\right)d{\tilde{G}}_{\tilde{\theta }}\left(x\right)=\int K^{†}\left(t,x/\tilde{\theta }\right)f_0\left(x\right)dx,$$

$$\psi \left(t,\mu \right)=\int K\left(t,y^{-1}{\left(1-\theta \right)}^{-1}\right)h_0\left(y\right){\nu }_{-1}\left(dy\right)=\left(K⊛h_0\right)\left(t,\mu \right),$$

$${\mathcal{D}}_1\left(t,\mu ;a,b\right)=\frac{1}{\pi }{\int }_{\left(\mu -b\right)t}^{\left(\mu -a\right)t}\frac{\sin y}{y}dy\text{ and }{\mathcal{D}}_2\left(t,\mu ;b\right)=\frac{1}{\pi }{\int }_0^t\frac{\sin \left\{\left(\mu -b\right)y\right\}}{y}dy$$

$$\mathcal{D}_{1,\infty}\left(\mu;a,b\right)=\lim_{t\rightarrow\infty}\mathcal{D}_{1}\left(t,\mu;a,b\right)=\left\{\begin{array}[c]{lll}1&\text{if}&a<\mu<b\\ 2^{-1}&\text{if}&\mu=a\text{ or }\mu=b\\ 0&\text{if}&\mu<a\text{ or }\mu>b\end{array}\right.,$$

$$\mathcal{D}_{2,\infty}\left(\mu;b\right)=\lim_{t\rightarrow\infty}\mathcal{D}_{2}\left(t,\mu;b\right)=\left\{\begin{array}[c]{lll}2^{-1}&\text{if}&\mu>b\\ 0&\text{if}&\mu=b\\ -2^{-1}&\text{if}&\mu<b\end{array}\right..$$

$${\psi }_{1,0}\left(t,\mu ;{\mu }_0\right)$$

$${\psi }_1\left(t,\mu \right)$$

$$={\mathcal{D}}_1\left(t,\mu ;a,b\right)=\frac{1}{2\pi }{\int }_{-t}^t\exp \left(\iota \mu s\right)\tilde{g}\left(s\right)ds,$$

$${\psi }_1=K_1\ast f_0={\mathcal{H}}_1\left\{1_{\left[-t,t\right]}\left(s\right)\times \left(\mathcal{H}{1}_{\left(a,b\right)}\left(y\right)\right)\left(s\right)\right\},$$

$${\psi }_1\left(t,\mu \right)$$

$$=\frac{1}{2\pi }{\int }_0^{t}dy{\int }_{-1}^1\mu \exp \left(\iota ys\mu \right)ds,$$

$${\psi }_1\left(t,\mu \right)=\left(K_1\ast f_0\right)\left(t,\mu \right)={\int }_0^t\mu \times \left({\mathcal{H}}_1{1}_{\left[-1,1\right]}\left(s\right)\right)\left(y\mu \right)dy.$$

$$K_1^{†}\left(t,x\right)=\frac{1}{2\pi }{\int }_0^{1}dy{\int }_{-1}^1\frac{1}{\iota y}\frac{d}{ds}\frac{\exp \left(\iota tysx\right)}{r_0\left(tys\right)}ds;$$

$$K_1\left(t,x\right)=\frac{t}{2\pi }{\int }_a^{b}dy{\int }_{\left[-1,1\right]}\frac{\cos \left\{ts\left(x-y\right)\right\}}{r_0\left(ts\right)}ds.$$

$${\psi }_1\left(t,\mu \right)=\int K_1\left(t,x\right)d{F}_{\mu }\left(x\right)=\frac{1}{\pi }{\int }_{\left(\mu -b\right)t}^{\left(\mu -a\right)t}\frac{\sin y}{y}dy,$$

$$\left\{\begin{array}[c]{l}K\left(t,x\right)=K_{1}\left(t,x\right)-2^{-1}\left\{K_{1,0}\left(t,x;a\right)+K_{1,0}\left(t,x;b\right)\right\}\\ \psi\left(t,\mu\right)=\psi_{1}\left(t,\mu\right)-2^{-1}\left\{\psi_{1,0}\left(t,\mu;a\right)+\psi_{1,0}\left(t,\mu;b\right)\right\}\end{array}\right..$$

Full text

Keywords: Composite null hypothesis; harmonic analysis; Lebesgue-Stieltjes integral equations; proportion of false null hypotheses

MSC 2010 subject classifications: Primary 62F12, 42A38; Secondary 45H05.

1 Introduction

The proportion of false null hypotheses and its dual, the proportion of true null hypotheses, play important roles in statistical modelling and in control and estimation of the false discovery rate (FDR,Benjamini and Hochberg (1995)) and the false nondiscovery rate (FNR, Genovese and Wasserman (2002)). For example, simultaneously testing or estimating the means of Gaussian random variables can be equivalently phrased as simultaneously testing or estimating the regression parameters of a linear model, and if we are interested in regression parameters whose values are within a specified range, then the proportion of such parameters or some other average norm of the regression parameters somehow characterizes how well an estimator of the parameter vector or a testing procedure on the parameters can perform; see, e.g., Abramovich et al. (2006). On the other hand, when simultaneously testing whether the means or medians of many test statistics are equal to a specific value or within a specific range, information on the proportion of such means or medias helps better control the FDR and FNR and may improve the power of a testing procedure, and estimators of the proportions are often used to construct “adaptive FDR procedures” and “adaptive FNR procedures” that are (asymptotically) conservative and often more powerful than their non-adaptive counterparts; see, e.g., Storey et al. (2004); Benjamini et al. (2006); Sarkar (2006); Hu et al. (2010); Nandi et al. (2021) for adaptive FDR procedures for testing a point null hypothesis. Further, these proportions are key components of the Bayesian two-component mixture model of Efron et al. (2001), its extensions Ploner et al. (2006); Cai and Sun (2009); Liu et al. (2016), and their corresponding decision rules such as the “local FDR” Efron et al. (2001), “q-value” Storey (2003) and the optimal discovery procedure Storey (2007). Without information on the proportions, these models and decision rules cannot be implemented in practice. Finally, these proportions can be used to asymptotically ensure that a certain fraction of false null hypotheses are rejected among a set of rejections, as done by Jin (2008). However, neither the proportion of false null hypotheses nor its dual is known, and it is very important to accurately estimate the proportions.

There are many estimators of the proportion of false (or true) null hypotheses; see, e.g., those of Storey (2002); Storey et al. (2004); Genovese and Wasserman (2004); Kumar and Bodhisattva (2016); Swanepoel (1999); Meinshausen and Rice (2006); Jin (2008); Jin and Cai (2007); Jin et al. (2010); Cai and Jin (2010); Chen (2019). However, they all estimate the proportion of parameters that are equal to a specific value, i.e., they are almost all for the setting of a point null hypothesis. Further, these estimators employ the Bayesian two-component mixture model (which then forces test statistics whose parameters are tested to be identically distributed marginally), require p-values to be identically distributed under the alternative hypothesis or uniformly distributed under the null hypothesis, require the distributions of test statistics to be members of a location-shift family, or are usually inconsistent. A more detailed discussion on these proportion estimators and others (including their underlying assumptions, advantages and disadvantages) is given by Chen (2019).

In contrast to the setting of a point null hypothesis, there are many situations where composite null hypotheses are tested and where the null hypothesis assumes that parameters being tested are greater than or smaller than a fixed value or are within a specific, finite range. For example, in gene expression studies, a biologist may only be interested in genes whose expression fold-changes exceed a threshold, or in genes that are down regulated or up regulated, whereas in studies of drug side effects, only drugs whose side effects are within a specific range can be recommended for (emergent) usage. These are related to two types of commonly used composite null hypotheses: “one-sided null”, i.e., a one-sided, unbounded interval to which the null parameter belongs, and “bounded null”, i.e., a non-empty, bounded interval to which the null parameter belongs. Note that a bounded null is a much more refined assessment on a parameter than a point null hypothesis. For multiple testing such composite hypotheses, it is quite useful to construct adaptive FDR and FNR procedures that employ consistent proportion estimators that are specifically designed for these hypotheses. However, such proportion estimators are scarce and often not consistent.

1.1 Review of major existing works

We will abbreviate “point null hypothesis” as “point null”, refer to an estimator of the proportion of true (or false) null hypotheses as a “null (or alternative) proportion estimator”, and take a frequentist perspective. First, there does not seem to be a proportion estimator that is designed for a one-side null. Storey’s estimator of Storey et al. (2004) and the “MR” estimator of Meinshausen and Rice (2006) were motivated by and designed for proportion estimation for a point null, are based on p-values, and require p-value to be uniformly distributed under the null hypothesis. In addition, the consistency of the “MR” estimator relies on the assumption that p-values under the alternative hypothesis are identically distributed. Doubtlessly, these two estimators can be applied to proportion estimation whenever p-values can be defined. Specifically, they can be applied to a one-sided or bounded null since the p-value for testing either of the nulls can be defined, e.g., by Definition 2.1 in Chapter of Dickhaus (2014). However, the p-value for testing a composite null is not uniformly distributed under the null hypothesis, and these two estimators are not theoretically guaranteed to function properly in this setting. Further, it seems natural to regard Storey’s estimator and the MR estimator as being inapplicable to proportion estimation for a bounded null due to the issues on defining a p-value for testing such a null.

Second, Jin’s estimator, which was constructed in Section 6 of Jin (2008), can be applied to a special case of a bounded null (i.e., a symmetric bounded null where the null parameter set is a symmetric interval around [math]) but is not applicable to a one-sided null. Specifically, this estimator was only constructed for Gaussian family via Fourier transform, and the construction fails for a one-sided null since the Fourier transform for the indicator function of an unbounded interval is undefined. We point out that Jin’s estimator can be used to estimate proportions induced by a suitable function of the magnitude of the mean parameter of Gaussian random variables, where the function has a compact support that is a symmetric interval around [math] and is even and continuous on its support; see Section 6 of Jin (2008) for more details on this. However, the consistency of these proportion estimators has not been proved in the sense of definition (2.6) to be introduced next.

Thirdly, there is another line of research that uses randomized p-values for composite nulls; see, e.g., Dickhaus (2013); Hoang and Dickhaus (2021b, a), who for a composite null first define p-value under the “least favorable parameter configurations (LFCs)” and then define randomized p-value by randomizing p-value under LFCs. With such randomized p-values for a composite null, proportion estimators that are based on p-values such as that of Schweder and Spjøtvoll (1982) can be used to estimate the proportion of true null hypotheses, and this was done by Hoang and Dickhaus (2021a). Note that the estimator of Schweder and Spjøtvoll (1982) implicitly requires p-values to be uniformly distributed under the true null hypothesis. However, the definition of randomized p-value for a composite null proposed by Dickhaus (2013); Hoang and Dickhaus (2021b) requires a specific set of assumptions to yield a valid p-value or a valid p-value that can be practically computed. Further, only for testing a one-side null on the means of Gaussian random variables with unit variance or testing the sign of the means of Student t random variables induced by Gaussian random variables, Dickhaus (2013); Hoang and Dickhaus (2021b, a) showed that their randomized p-values are valid and provided methods on how to compute them, whereas for a bounded null they provided no statistical models for which their definition of randomized p-value leads to valid p-value or its computation in practice. Regardless, to use randomized p-value proposed by these authors, we need to check whether needed assumptions are satisfied for each statistical model we employ and devise an algorithm to compute such p-value in practice correspondingly.

1.2 Main contributions and summary of results

In this work, we employ the frequentist paradigm as a complement to the Bayesian one, take the parameter as the mean or median of a random variable, and consider proportion estimation for multiple testing one-sided nulls, bounded nulls, and a suitable functional of a bounded null on the parameters, respectively. For each multiple testing scenario, we construct uniformly consistent proportion estimators as solutions to Lebesgue-Stieltjes integral equations and via Fourier analysis and Mellin transform. These new estimators are not based on p-values, and they do not require random variables, test statistics or p-values to be identically distributed marginally, identically distributed under the alternative hypothesis, or uniformly distributed under the null hypothesis. Since both bounded and one-sided nulls have finite boundary points, estimators of Chen (2019) that are for a point null are used to deal with proportions related solely to these points. However, we provide new constructions of proportion estimators that utilize Dirichlet integral to approximate the indicator function of a bounded or one-sided null. These constructions are quite different from those in Jin (2008); Jin and Cai (2007); Jin et al. (2010); Cai and Jin (2010); Chen (2019), and further demonstrate the power of integral equations and harmonic analysis in statistical estimation.

Our main contributions are summarized as follows:

• •

“Construction I”: Construction of proportion estimators for testing a bounded null on the means or medians of random variables whose distributions are members of a Type I location-shift family (see Definition 2.1) and the Gamma family respectively, together with their speeds of convergence and uniform consistency classes for the Gaussian family and the Gamma family respectively.

• •

“Construction II”: Construction of proportion estimators for testing a one-sided null on the means or medians of random variables whose distributions are members of a Type I location-shift family and the Gamma family respectively, together with their speeds of convergence and uniform consistency classes for the Gaussian family and the Gamma family respectively.

• •

Extension of Construction I to estimate the “proportions” induced by a function of a bounded null that is continuous and of bounded variation, together with their speeds of convergence and uniform consistency classes for the Gaussian family and the Gamma family respectively. This considerably extends the constructions in Section 6 of Jin (2008) and strengthens Theorem 13 there.

Our new estimators are the first uniformly consistent proportion estimators for a one-sided null or a bounded null in the literature, and do not rely on a concept of p-value for testing a composite null. Note here “consistency” is defined via the “ratio” rather than the “difference” between an estimator and the alternative proportion, in order to account for a diminishing alternative proportion; see the definition in (2.6). For estimating the proportion of false null hypotheses under independence for both bounded and one-sided nulls, the maximal speeds of convergence of our new estimators are of the same order as those for the proportion estimators for a point null in Chen (2019). Specifically, the maximal speeds have the same order as $\sqrt{\ln m}$ for Gaussian family and approximately have the same order as $\ln m$ for the Gamma family, where $m$ is the number of hypotheses to test. However, due to the use of Dirichlet integral to approximate the indicator function of a bounded or one-sided null, the sparsest alternative proportions the new estimators can consistently estimate under independence are of smaller order than their maximal speeds of convergence. In contrast, the sparsest alternative proportion an estimator for a point null in Chen (2019) and Jin (2008) can consistently estimate under independence can be of the same order as $m^{\varepsilon-1/2}$ for any $\varepsilon\in\left(0,0.5\right]$.

Such a difference between the sparsest alternative proportions that these estimators can consistently estimate under a point null, bounded null or one-sided null respectively is the main consequence of the speed of convergence of the Oracle estimator (to be introduced in Section 2.2), and this speed is related to that of Dirichlet integral for a one-sided or bounded null but is independent of the alternative proportion for a point null. In other words, we have discovered the universal phenomenon that, for an alternative proportion estimator that is constructed via a solution to some Lebesgue-Stieltjes equation as an approximator to the indicator of (a transform of) the parameter set under the alternative hypothesis, the sparsest alternative proportion such an estimator is able to consistently estimate can never be of larger order than the maximal speed of convergence of the solution to its targeted indicator function.

As a by-product, we provide the speed of convergence of Dirichlet-type integrals (see Lemma A.2 and Lemma 6.1), upper bounds on the moments of Gamma distributions (see Lemma E.1), and a classification result on natural exponential family (see Theorem A.2), which are of independent interest. Our simulation studies show that the new estimators often perform much better than the MR estimator, Storey’s estimator, and proportion estimators of Hoang and Dickhaus (2021b, a) that are based on randomized p-values.

1.3 Notations and conventions

The notations and conventions we will use throughout are stated as follows: $C$ denotes a generic, positive constant whose values may differ at different occurrences; $O\left(\cdot\right)$ and $o\left(\cdot\right)$ are respectively Landau’s big O and small o notations; $\mathbb{E}$, $\mathbb{V}$ and $\mathsf{cov}$ are respectively the expectation, variance and covariance with respect to the probability measure $\Pr$; $\mathbb{R}$ and $\mathbb{C}$ are respectively the set of real and complex numbers; $\Re$ denotes the real part of a complex number; $\mathbb{N}$ denotes the set of non-negative integers, and $\mathbb{N}_{+}=\mathbb{N}\setminus\left\{0\right\}$; $\delta_{y}$ is the Dirac mass at $y\in\mathbb{R}$; $\nu$ the Lebesgue measure, and when no confusion arises, the usual notation $d\cdot$ for differential will be used in place of $\nu\left(d\cdot\right)$; for a real-valued (measurable) function $f$ defined on some (measurable) $A\subseteq\mathbb{R}$, $\left\|f\right\|_{p}=\left\{\int_{A}\left|f\left(x\right)\right|^{p}\nu\left(dx\right)\right\}^{1/p}$ and $L^{p}\left(A\right)=\left\{f:\left\|f\right\|_{p}<\infty\right\}$ for $1\leq p\leq\infty$, $\left\|f\right\|_{\infty}$ is its essential supremum, and $\left\|f\right\|_{\mathrm{TV}}$ is the total variation of $f$ on $A$; for a set $A\subseteq\mathbb{R}^{d}$, $\left|A\right|$ is the cardinality of $A$, and $1_{A}$ the indicator of $A$; $\partial_{\cdot}$ denotes the derivative with respect to the subscript; $\mathbb{R}^{\mathbb{N}}$ is the $\aleph$-Cartesian product of $\mathbb{R}$, where $\aleph$ is the cardinality of $\mathbb{N}$; $\left[x\right]$ denotes the integer part of a real number $x$; $\mathcal{N}_{m}\left(\mathbf{u},\mathbf{S}\right)$ denotes the density and distribution of the $m$-dimensional Gaussian random vector with mean vector $\mathbf{u}$ and covariance matrix $\mathbf{S}$, and $\Phi$ the cumulative distribution function (CDF) of $X\sim\mathcal{N}_{1}\left(0,1\right)$.

1.4 Organization of article

The rest of the article is organized as follows. We formulate in Section 2 the problem of proportion estimation and provide the needed background, and give in Section 3 an overview of the main constructions used or proposed here. We develop in Section 4 uniformly consistent proportion estimators for multiple testing the means or medians of random variables from a Type I location-shift family and in Section 5 such estimators for multiple testing the means of random variables from the Gamma family, and extend in Section 6 these constructions to estimate “proportions” induced by a continuous function of a bounded null that is of bounded variation. We end the article with a discussion in Section 7. All proofs, some auxiliary results, and simulation studies on the proposed estimators are given in the supplementary material.

2 Preliminaries

We formulate in Section 2.1 the problem of proportion estimation and in Section 2.2 the strategy to proportion estimation via Lebesgue-Stieltjes integral equations which further generalizes that in Chen (2019), provide in Section 2.3 a very brief background on location-shift families and the Gamma family, and present in Section 2.4 the key results of Chen (2019) on proportion estimation for the setting of a point null that are needed here for the constructions of proportion estimators for the setting of composite nulls.

2.1 The estimation problem

Let $\Theta_{0}$ be a subset of $\mathbb{R}$ that has a non-empty interior and non-empty complement $\Theta_{1}=\mathbb{R}\setminus\Theta_{0}$. For each $i\in\left\{1,\ldots,m\right\}$, let $z_{i}$ be a random variable with mean or median $\mu_{i}$, such that, for some integer $m_{0}$ between [math] and $m$, $\mu_{i}\in\Theta_{0}$ for each $i\in\left\{1,\ldots,m_{0}\right\}$ and $\mu_{i}\in\Theta_{1}$ for each $i\in\left\{m_{0}+1,\ldots,m\right\}$. Consider simultaneously testing the null hypothesis $H_{i0}:\mu_{i}\in\Theta_{0}$ versus the alternative hypothesis $H_{i1}:\mu_{i}\in\Theta_{1}$ for all $i\in\left\{1,\ldots,m\right\}$. Let $I_{0,m}$ $=\left\{i\in\left\{1,\ldots,m\right\}:\mu_{i}\in\Theta_{0}\right\}$ and $I_{1,m}=\left\{i\in\left\{1,\ldots,m\right\}:\mu_{i}\in\Theta_{1}\right\}$. Then the cardinality of $I_{0,m}$ is $m_{0}$, the proportion of true null hypothesis (“null proportion” for short) is defined as $\pi_{0,m}=m^{-1}m_{0}$, and the proportion of false null hypotheses (“alternative proportion” for short) $\pi_{1,m}=1-\pi_{0,m}$. In other words,

[TABLE]

is the proportion of random variables whose means or medians are in $\Theta_{1}$. Our target is to consistently estimate $\pi_{1,m}$ as $m\rightarrow\infty$, and we will focus on the “bounded null” $\Theta_{0}=\left(a,b\right)$ for some fixed, finite $a,b\in U$ with $a<b$ and the “one-sided null” $\Theta_{0}=\left(-\infty,b\right)$, both of which are composite nulls. However, the strategy to be introduced next in Section 2.2 to achieve this target applies to general $\Theta_{0}$ (and hence general $\Theta_{1}$).

2.2 The strategy via solutions to Lebesgue-Stieltjes integral

equations

Let $\mathbf{z}=\left(z_{1},\ldots,z_{m}\right)^{\top}$ and $\boldsymbol{\mu}=\left(\mu_{1},...,\mu_{m}\right)^{\top}$. Denote by $F_{\mu_{i}}$ the CDF of $z_{i}$ for $i\in\left\{1,\ldots,m\right\}$ and suppose each $F_{\mu_{i}}$ is a member of a set $\mathcal{F}$ of CDFs such that $\mathcal{F}=\left\{F_{\mu}:\mu\in U\right\}$ for some non-empty $U$ in $\mathbb{R}$. For the rest of the paper, we assume that each $F_{\mu}$ is uniquely determined by $\mu$ and that $U$ has a non-empty interior. Recall the definition of $\pi_{1,m}$ in (2.1). Then $\pi_{1,m}=m^{-1}\sum_{i=1}^{m}1_{\Theta_{1}}\left(\mu_{i}\right)$. The strategy to estimate $\pi_{1,m}$ via Lebesgue-Stieltjes integral equations approximates each indicator function $1_{\Theta_{1}}\left(\mu_{i}\right)$, and is stated below.

Suppose for each fixed $\mu\in U$, we can approximate the indicator function $1_{\Theta_{0}}\left(\mu\right)$ by

• C1)

A “discriminant function” $\psi\left(t,\mu\right)$ satisfying $\lim_{t\rightarrow\infty}\psi\left(t,\mu\right)=1$ for $\mu\in\Theta_{0}$ and $\lim_{t\rightarrow\infty}\psi\left(t,\mu\right)=0$ for $\mu\in\Theta_{1}$, and

• C2)

A “matching function” $K:\mathbb{R}^{2}\rightarrow\mathbb{R}$ that does not depend on any $\mu\in\Theta_{1}$ and satisfies the Lebesgue-Stieltjes integral equation

[TABLE]

Then the “average discriminant function”

[TABLE]

satisfies $\lim_{t\rightarrow\infty}\varphi_{m}\left(t,\boldsymbol{\mu}\right)=\pi_{1,m}$ for any fixed $m$ and $\boldsymbol{\mu}$, and provides the “Oracle” $\Lambda_{m}\left(\boldsymbol{\mu}\right)=\lim_{t\rightarrow\infty}\varphi_{m}\left(t,\boldsymbol{\mu}\right)$ for each fixed $m$ and $\boldsymbol{\mu}$. Further, the “empirical matching function”

[TABLE]

satisfies $\mathbb{E}\left\{\hat{\varphi}_{m}\left(t,\mathbf{z}\right)\right\}=\varphi_{m}\left(t,\boldsymbol{\mu}\right)$ for any fixed $m,t$ and $\boldsymbol{\mu}$. Namely, $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is an unbiased estimator of $\varphi_{m}\left(t,\boldsymbol{\mu}\right)$. We will reserve the notation $\left(\psi,K\right)$ for a pair of discriminant function and matching function and the notations $\varphi_{m}\left(t,\boldsymbol{\mu}\right)$ and $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ as per (2.3) and (2.4) unless otherwise noted. The concept of discriminant function and matching function originates from, is inspired by, and extends the concept of “phase functions” in the work of Jin (2008). The pair $\left(\psi,K\right)$ presented here has those in Chen (2019) and Jin (2008) as special cases, and is the most general form for the purpose of proportion estimation. It converts proportion estimation into solving a specific Lebesgue-Stieltjes integral equation.

When the difference

[TABLE]

is small for large $t$, $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ will accurately estimate $\pi_{1,m}$. Since $\varphi_{m}\left(t,\boldsymbol{\mu}\right)=\pi_{1,m}$ or $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)=\pi_{1,m}$ rarely happens, $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ usually employs an increasing sequence $\left\{t_{m}\right\}_{m\geq 1}$ with $\lim_{m\rightarrow\infty}t_{m}=\infty$ in order to achieve consistency in the sense that

[TABLE]

Following the convention set by Chen (2019), we refer to $t_{m}$ as the “speed of convergence” of $\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)$. Throughout the paper, consistency of a proportion estimator is defined via (2.6) to accommodate the scenario $\lim_{m\rightarrow\infty}\pi_{1,m}=0$. Further, the accuracy of $\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)$ in terms of estimating $\pi_{1,m}$ and its speed of convergence depend on how fast $\pi_{1,m}^{-1}e_{m}\left(t_{m}\right)$ converges to [math] and how fast $\pi_{1,m}^{-1}\varphi_{m}\left(t_{m},\boldsymbol{\mu}\right)$ converges to $1$. This general principle also applies to the works of Jin (2008); Jin and Cai (2007) and Chen (2019).

By duality, $\varphi_{m}^{\ast}\left(t,\boldsymbol{\mu}\right)=1-\varphi_{m}\left(t,\boldsymbol{\mu}\right)$ satisfies $\pi_{0,m}=\lim_{t\rightarrow\infty}\varphi_{m}^{\ast}\left(t,\boldsymbol{\mu}\right)$ for any fixed $m$ and $\boldsymbol{\mu}$, and $\hat{\varphi}_{m}^{\ast}\left(t,\mathbf{z}\right)=1-\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ satisfies $\mathbb{E}\left\{\hat{\varphi}_{m}^{\ast}\left(t,\mathbf{z}\right)\right\}=\varphi_{m}^{\ast}\left(t,\boldsymbol{\mu}\right)$ for any fixed $m,t$ and $\boldsymbol{\mu}$. Moreover, $\hat{\varphi}_{m}^{\ast}\left(t,\mathbf{z}\right)$ will accurately estimate $\pi_{0,m}$ when $e_{m}\left(t\right)$ is suitably small for large $t$, and the stochastic oscillations of $\hat{\varphi}_{m}^{\ast}\left(t,\mathbf{z}\right)$ and $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ are the same and is quantified by $e_{m}\left(t\right)$.

2.3 Type I location-shift family and Gamma family

Recall $\mathcal{F}=\left\{F_{\mu}:\mu\in U\right\}$ and that the CDF $F_{\mu_{i}}$ of $z_{i}$ for each $i$ is a member of $\mathcal{F}$. First, let us discuss the setting where $\mathcal{F}$ is a location-shift family since it is widely used. Recall the definition of location-shift family, i.e., $\mathcal{F}$ is a location-shift family if and only if $z+\mu^{\prime}$ has CDF $F_{\mu+\mu^{\prime}}$ whenever $z$ has CDF $F_{\mu}$ for $\mu,\mu+\mu^{\prime}\in U$. Let $\hat{F}_{\mu}\left(t\right)=\int e^{\iota tx}dF_{\mu}\left(x\right)$ be the characteristic function (CF) of $F_{\mu}$ where $\iota=\sqrt{-1}$. We can write $\hat{F}_{\mu}=r_{\mu}e^{\iota h_{\mu}}$, where $r_{\mu}$ is the modulus of $\hat{F}_{\mu}$ and $h_{\mu}$ is the argument of $\hat{F}_{\mu}$ (to be determined case-wise). If $\mathcal{F}$ is a location-shift family, then $\hat{F}_{\mu}\left(t\right)=\hat{F}_{\mu_{0}}\left(t\right)\exp\left\{\iota t\left(\mu-\mu_{0}\right)\right\}$ for all $\mu,\mu_{0}\in U$ and $r_{\mu}$ does not depend on $\mu$. In order to construct proportion estimators for a one-sided or bounded null or a functional of a bounded null using the strategy in Section 2.2 when $\mathcal{F}$ is a location-shift family, we need

Definition 2.1**.**

$\mathcal{F}$* is a “Type I location-shift family” if $\mathcal{F}$ is a location-shift family for which $\hat{F}_{0}$ has no real zeros and $\hat{F}_{0}=r_{0}$.*

With respect to Definition 2.1, we have a few remarks ready. First, $\hat{F}_{0}=r_{0}$ holds if $h_{0}\equiv 0$, and it implies $h_{\mu}\left(t\right)=\mu t$, $\hat{F}_{\mu}\left(t\right)=\hat{F}_{0}\left(t\right)\exp\left(\iota t\mu\right)$ and that, if $F_{\mu}$ has a density function $f_{\mu}$, then $f_{\mu}$ is an even function, for all $\mu\in U$ and $t\in\mathbb{R}$. In particular, $\hat{F}_{0}=r_{0}$ if and only if the CF of $F_{0}$ is real, even and non-negative. Second, the requirement “$\hat{F}_{0}$ has no real zeros” facilitates division by $\hat{F}_{0}$ to construct matching functions $K$. Note that if $\hat{F}_{0}$ has no real zeros, then $\hat{F}_{\mu}$ has no real zeros for all $\mu\in U$. Since $r_{\mu}\equiv r_{0}$ for all $\mu\in U$ for any location-shift family, then $r_{\mu}(t)/r_{0}(t)=1$ for all $\mu\in U$ and all $t\in\mathbb{R}$ whenever $r_{0}$ has no real zeros. The condition “$\sup_{t\in\mathbb{R}}r_{\mu}(t)/r_{0}(t)<\infty$ for each $\mu\in U$” ensures the validity of the generalized Riemann-Lebesgue lemma as Theorem 3 of Costin et al. (2016) to construct $K$. Namely, Definition 2.1 is a special case of Definition 1 of Chen (2019) that states “$\mathcal{F}$ is a location-shift family with Riemann-Lebesgue type characteristic functions (RL-CFs)” and that is needed to construct matching functions for a point null there. Third, the requirement “$\hat{F}_{0}$ has no real zeros and $\hat{F}_{0}=r_{0}$” is not restrictive since it is satisfied by many location-shift families used in practice, which include Gaussian family and four other families to be provided in Section 4.3. Since enumerating all Type I location-shift families is equivalent to the open problem of classifying real, even functions on $\mathbb{R}$ whose Fourier transforms are positive (which will not be pursued here; see, e.g., Giraud and Peschanski (2014); Tuck (2006)), we will only provide in Section A.1 two methods to construct these families.

As revealed by Jin (2008); Chen (2019) and will be seen later, the key appeal of a Type I location-shift family in constructing proportion estimators for both a one-sided null and a bounded null respectively is that its location-shift property utilizes the additive structure of the group $\left(\mathbb{R},+\right)$, i.e., $\mathbb{R}$ with addition “$+$”, and couples very well with convolution on an additive group, Fourier transform, and Dirichlet integrals to provide solutions to the Lebesgue-Stieltjes integral equation (2.2). On the other hand, $\mathcal{F}$ may not be a location-shift family but possesses some “scaling-invariance property” that is also revealed by its moment functions. One such $\mathcal{F}$ is the Gamma family.

The Gamma family $\mathcal{F}=\mathsf{Gamma}\left(\theta,\sigma\right)$ has generating measure (called “basis”) $\beta$ such that

[TABLE]

where $\Gamma$ is the Gamma function. So, this family of CDFs is indexed by its natural parameter $\theta\in\Theta=\{\theta:\theta<1\}$, contains CDF $G_{\theta},\theta\in\Theta$ which has density

[TABLE]

and has mean function $\mu=\mu\left(\theta\right)=\int_{0}^{\infty}xf_{\theta}\left(x\right)dx=\sigma\left(1-\theta\right)^{-1}$ with $\mu\in U=\mu\left(\Theta\right)$. Note that $f_{0}=\frac{d\beta}{d\nu}$. If we set $\tilde{\theta}=1-\theta$, then we can write $f_{\theta}\left(x\right)$ more compactly as

[TABLE]

and write its CDF as $\tilde{G}_{\tilde{\theta}}$. Note that $\tilde{f}_{\tilde{\theta}}\left(x\right)=f_{\theta}\left(x\right)$ and $G_{\theta}=$ $\tilde{G}_{\tilde{\theta}}$. With these notational adjustments, we quickly see that the Gamma family has the following “scaling-invariance property”: if $X$ has CDF $\tilde{G}_{\tilde{\theta}}$, then $Y=\tilde{\theta}X$ has CDF $\beta$. This is similar to the “location-invariance property” for a location-shift family $\mathcal{F}$ that, if $X$ has CDF $F_{\mu}$, then $Y=X-\mu$ has CDF $F_{0}$, where $F_{0}$ can be taken as the “basis” that generates all $F_{\mu}\in\mathcal{F}$. The key difference is that for a location-shift family we use the additive group $\left(\mathbb{R},+\right)$ and its convolution with respect to $+$, whereas for the Gamma family, we use the multiplicative group $\left(\mathbb{R}_{>0},\times\right)$, i.e., the set $\mathbb{R}_{>0}$ of positive real numbers with multiplication “$\times$”, and its convolution with respect to $\times$.

The Gamma family has moment functions

[TABLE]

where we recognize that $\int x^{z}G_{\theta}\left(dx\right),z\in\mathbb{C}$ is the Mellin transform of $G_{\theta}$. We see $\tilde{c}_{n}\left(\theta\right)=\xi^{n}\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{n}$ for each $n\in\mathbb{N}$ and $\theta\in\Theta$, where $\xi\left(\theta\right)=\left(1-\theta\right)^{-1}$, $\tilde{a}_{n}=\Gamma\left(n+\sigma\right)\left\{\Gamma\left(\sigma\right)\right\}^{-1}$ and $\zeta\left(\theta\right)=\zeta_{0}$ for the constant $\zeta_{0}=1$. The special structure for the moment functions $\tilde{c}_{n}$ is a consequence of the scaling-invariance property of the Gamma family. Further, $\tilde{a}_{1}=\sigma$, $f_{\theta}\left(x\right)=O\left(x^{\sigma-1}\right)$ as $x\rightarrow 0+$, and $\Psi\left(t,\theta\right)=\sum_{n=0}^{\infty}\frac{t^{n}\xi^{n}\left(\theta\right)}{\tilde{a}_{n}n!}$ is absolutely convergent pointwise in $\left(t,\theta\right)\in\mathbb{R}\times\Theta$. Note that $\mu\left(\theta\right)=\xi\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{1}$, that $\mu=\mu\left(\theta\right)$ has a unique inverse function $\theta=\theta(\mu)\in\Theta$, and that each $G_{\theta}$ corresponds uniquely to a non-degenerate CDF $F_{\mu}$ and vice versa, i.e., $\mathcal{F}=\{G_{\theta}:\theta\in\Theta\}=\{F_{\mu}:\mu\in U\}$.

For random variables whose CDFs are members of the Gamma family, testing their means $\mu_{i}$’s simultaneously is equivalent to testing natural parameters $\theta_{i}=\theta\left(\mu_{i}\right)$ simultaneously. So, for the constructions of proportion estimators via the strategy in Section 2.2, $K$ and $\psi$ will also be regarded as functions of $\theta$. Specifically, $\psi$ defined by (2.2) becomes

[TABLE]

Let $\boldsymbol{\theta}=\left(\theta_{1},\ldots,\theta_{m}\right)^{\top}$. Then accordingly $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}\left\{1-K\left(t,z_{i}\right)\right\}$ and $\varphi_{m}\left(t,\boldsymbol{\theta}\right)=m^{-1}\sum_{i=1}^{m}\left\{1-\psi\left(t,\theta_{i}\right)\right\}$ become the counterparts of (2.4) and (2.3).

The Gamma family is a natural exponential family (NEF) that contains the family of exponential distributions and the family of central Chi-square distributions, and it has a sequence of very special moment functions $\{\tilde{c}_{n},n\geq 0\}$ such that $\tilde{c}_{n}\left(\theta\right)=\xi^{n}\left(\theta\right)\tilde{a}_{n}$ for each $n\in\mathbb{N}$ and $\theta\in\Theta$ (given earlier), which we refer to as “separable moments”. Appendix A provides a very short background on NEFs and Theorem A.2 there shows that the only NEF with separable moments is the Gamma family. As revealed by Chen (2019) and will be seen later, the key appeal of the Gamma family in constructing proportion estimators for both a one-sided null and a bounded null respectively is that its scaling-invariant property utilizes the multiplicative structure of the group $\left(\mathbb{R}_{>0},\times\right)$, and couples well with convolution on a multiplicative group, Mellin transform, and Dirichlet integrals to provide solutions to the Lebesgue-Stieltjes integral equation (2.2). Beyond the case of Type I location-shift family or Gamma family, we have not been able to find a non-trivial solution $\mathcal{F}=\{F_{\mu}:\mu\in U\}$ to (2.2) for a bounded null or a one-side null despite considerable attempts.

2.4 Constructions of proportion estimators for a point null

Since the bounded null $\Theta_{0}=\left(a,b\right)$ and the one-sided null $\Theta_{0}=\left(-\infty,b\right)$ have two finite boundary points $a$ and $b$, in order to consistently estimate

[TABLE]

using the strategy in Section 2.2 for these two composite hypotheses, we can use the proportion estimators of Chen (2019) for a point null to specifically account for the proportion of $\mu_{i}$’s that are equal to $a$ or $b$ when $\mathcal{F}$ is a location-shift family or $\theta_{i}$’s that are equal to $\theta\left(a\right)$ or $\theta\left(b\right)$ when $\mathcal{F}$ is the Gamma family. Consider a point null $\Theta_{0}=\left\{\mu_{0}\right\}$ or $\left\{\theta_{0}\right\}$ for a fixed $\mu_{0}\in U$ or $\theta_{0}\in\Theta$. We can state the constructions of Chen (2019) of uniformly consistent estimators of $\pi_{1,m}$ for the point null when $\mathcal{F}$ is a location-shift family with RL-CFs or the Gamma family as follows in terms of a discriminant function and a matching function:

Theorem 2.1**.**

Let $\omega$ be an even, bounded, probability density function on $\left[-1,1\right]$. For $\mu^{\prime}\in U$ and $\theta^{\prime}\in\Theta$, define

[TABLE]

and let

[TABLE]

Then

[TABLE]

For the point null $\Theta_{0}=\left\{\mu_{0}\right\}$ with $\mu_{0}\in U$, $\left(\psi,K\right)=\left(\psi_{1,0}\left(t,\mu;\mu_{0}\right),K_{1,0}\left(t,x;\mu_{0}\right)\right)$ when $\mathcal{F}$ is a Type I location-shift family, whereas for the point null $\Theta_{0}=\left\{\theta_{0}\right\}$ with $\theta_{0}\in\Theta$, $\left(\psi,K\right)=\left(\psi_{3,0}\left(t,\theta;\theta_{0}\right),K_{3,0}\left(t,x;\theta_{0}\right)\right)$ when $\mathcal{F}$ is the Gamma family. In particular, $\psi_{1,0}\left(t,\mu_{0};\mu_{0}\right)=1$ for all $t$, and $\psi_{3,0}\left(t,\theta_{0};\theta_{0}\right)=1$ for all $t$.

Note that, by Riemann-Lebesgue Lemma, $\lim_{t\to\infty}\psi_{1,0}\left(t,\mu;\mu^{\prime}\right)=0$ for all $\mu\neq\mu^{\prime}$ and $\mu,\mu^{\prime}\in U$, and $\lim_{t\to\infty}\psi_{3,0}\left(t,\theta;\theta^{\prime}\right)=0$ for all $\theta\neq\theta^{\prime}$ and $\theta,\theta^{\prime}\in\Theta$. In this work, we will further assume that $\omega$ is of bounded variation unless otherwise noted. For example, the triangular density $\omega\left(s\right)=\left(1-|s|\right)1_{\left[-1,1\right]}\left(s\right)$ or the uniform density $\omega\left(s\right)=0.5\times 1_{\left[-1,1\right]}\left(s\right)$ can be used. Theorem 2.1 will be used by the constructions to be introduced in Section 4, Section 5 and Section 6.

3 Overview and illustration of constructions

We give an overview on the main constructions provided in Section 2.4 and to be introduced in later sections. Even though conceptually these constructions can be more elegantly stated and better understood in terms of complex analysis (as their proofs reveal), we describe them using terms of real analysis wherever feasible. Let $\ast$ denote the additive convolution with respect to $\left(\mathbb{R},+\right)$, and $\circledast$ the multiplicative convolution with respect to $\left(\mathbb{R}_{>0},\times\right)$. The constructions of the matching function $K$ for the Gamma family is much more subtle than those of $K$ for a location-shift family, even though the discriminant function $\psi$ for both the location-shift family and Gamma family has the same analytic expressions, where $\psi$ involves the additive convolution on $\left(\mathbb{R},+\right)$ for a location-shift family but the multiplicative convolution on $\left(\mathbb{R}_{>0},\times\right)$ for the Gamma family. So, we will only briefly describe the ideas behind constructing $K$ for the Gamma family and elaborate more on constructing $K$ for a Type I location-shift family.

Specifically, for the Gamma family, $K$ is the real part of $K^{\dagger}$, the integral of the product of the exponential function $\exp$ (on the complex domain) with a first set of multiplicative arguments and an exponential-type power series (on the complex domain) with a second set of multiplicative arguments that is related those of the first set, so that when $K^{\dagger}$ is integrated with respect to the measure $G_{\theta}$, i.e., when we define $\psi^{\dagger}\left(t,\mu\right)=\int K^{\dagger}\left(t,x\right)dG_{\theta}\left(x\right)$ and compute $\psi^{\dagger}\left(t,\mu\right)$ as

[TABLE]

the power series in $K^{\dagger}$ invokes Mellin transform of $G_{\theta}$, produces the exponential function $\exp$ (on the complex domain) with a third set of multiplicative arguments, and gives $\psi^{\dagger}$ as the integral of the product of two exponential functions $\exp$ respectively with two sets of interrelated multiplicative arguments. Since the exponential function $\exp$ is a map from the additive group $\left(\mathbb{C},+\right)$ of complex numbers to the multiplicative group $\left(\mathbb{C}_{\neq 0},\times\right)$ of nonzero complex numbers, where $\mathbb{C}_{\neq 0}=\left\{z\in\mathbb{C}:z\neq 0\right\}$, we obtain $\psi^{\dagger}$ whose real part $\psi$ is the desired discriminant function. Since (3.1) is not the multiplicative convolution $K^{\dagger}\circledast f_{0}$ with respect to the Lebesgue measure $\nu$, Fourier analysis with respect to $\nu$ and $\times$ cannot be applied to obtain $K^{\dagger}$ or $K$. However, since $f_{0}\left(x\right)dx=-h_{0}\left(y\right)\nu_{-1}\left(dy\right)$ with $y=x^{-1}$ for $x\in\mathbb{R}_{>0}$, where $h_{0}\left(y\right)=y^{-\sigma}e^{-1/y}\left\{\Gamma\left(\sigma\right)\right\}^{-1}1_{\left(0,\infty\right)}\left(y\right)$ and $\nu_{-1}\left(A\right)=\int_{A}x^{-1}dx$ for measurable $A\subseteq\mathbb{R}_{>0}$ is the Haar measure on the group $\left(\mathbb{R}_{>0},\times\right)$, we have

[TABLE]

and $K$ may be found via Fourier analysis with respect to $\nu_{-1}$ on $\left(\mathbb{R}_{>0},\times\right)$. In this setting, Mellin transform with pure imaginary argument $-\iota t,t\in\mathbb{R}$ corresponds to Fourier transform. The constructions described at the beginning of this paragraph are consistent with this perspective.

3.1 The case of Type I location-shift family

Now let us move on to the constructions for a Type I location-shift family $\mathcal{F}$. There are several conventions on the definition of Fourier transform $\mathcal{H}$ on the group $\left(\mathbb{R},+\right)$, and here we use $\left(\mathcal{H}f\right)\left(t\right)=\int e^{-\iota tx}f\left(x\right)dx$ and define $\mathcal{H}_{1}$ as $\left(\mathcal{H}_{1}f\right)\left(t\right)=\left(2\pi\right)^{-1}\int f\left(x\right)e^{\iota tx}dx$, whenever the integrals are defined. If needed, for $\mathcal{H}f$ or $\mathcal{H}_{1}f$, we will write out the argument of $f$ that is integrated out by $\mathcal{H}$ or $\mathcal{H}_{1}$. Also, we need two Dirichlet integrals

[TABLE]

and

[TABLE]

and

[TABLE]

Let $f_{\mu}$ be the density of $F_{\mu}$ with respect to the Lebesgue measure. Since $\mathcal{F}$ is a Type I location-shift family, then $f_{0}$ is an even function, $\hat{F}_{0}\equiv r_{0}$, and $\hat{F}_{\mu}$ has no real zeros for all $\mu\in U$. For the point null $\Theta=\left\{\mu_{0}\right\}$, the pair $\left(\psi_{1,0},K_{1,0}\right)$ given by Theorem 2.1 satisfies the following two identities: $\psi_{1,0}\left(t,\mu;\mu_{0}\right)=\left(\mathcal{H}\omega\right)\left(t\left(\mu-\mu_{0}\right)\right)$, i.e., $\psi_{1,0}\left(t,\mu;\mu_{0}\right)$ is the Fourier transform of $\omega$ evaluated at $t\left(\mu-\mu_{0}\right)$, and

[TABLE]

where the second identity only uses the location-shift property of $\mathcal{F}$. Since $f_{0}$ is even, (3.4) implies $\psi_{1,0}=K_{1,0}\ast f_{0}=\hat{\omega}$, and $K_{1,0}$ is the inverse Fourier transform of $\omega/\mathcal{H}f_{0}$ or of $\mathcal{H}\psi_{1,0}/\mathcal{H}f_{0}$.

For the bounded null $\Theta=\left(a,b\right)$, we first let $\tilde{g}\left(t\right)=\int_{a}^{b}\exp\left(-\iota yt\right)dy$, i.e., $\tilde{g}$ is the Fourier transform of the indicator function $1_{\left(a,b\right)}\left(y\right)$ of $\left(a,b\right)$, and then find $K_{1}\left(t,x\right)$ such that

[TABLE]

where the second identity uses the location-shift property of $\mathcal{F}$, and the last is proved by Lemma A.3 in Appendix A. Again since $f_{0}$ even, (3.5) implies

[TABLE]

and $K_{1}$ is the inverse Fourier transform of $\mathcal{H}\psi_{1}/\mathcal{H}f_{0}$. Now, if we set $\psi\left(t,\mu\right)=\psi_{1}\left(t,\mu\right)-2^{-1}\left\{\psi_{1,0}\left(t,\mu;a\right)+\psi_{1,0}\left(t,\mu;b\right)\right\}$, then $\lim_{t\rightarrow\infty}\psi\left(t,\mu\right)=1_{\left(a,b\right)}\left(\mu\right)$ due to (3.2). So, if we set $K\left(t,x\right)=K_{1}\left(t,x\right)-2^{-1}\left\{K_{1,0}\left(t,x;a\right)+K_{1,0}\left(t,x;b\right)\right\}$, then the pair $\left(\psi,K\right)$ solves the Lebesgue-Stieltjes equation (2.2).

For the one-sided null $\Theta=\left(-\infty,b\right)$ with some fixed, finite $b$, we can assume $b=0$ without loss of generality. Then we need to find $K_{1}\left(t,x\right)$ such that

[TABLE]

where the second identity uses the location-shift property of $\mathcal{F}$, and the fourth is proved by Lemma A.3 in Appendix A. Since $f_{0}$ is even, then

[TABLE]

However, due to (3.3) and Lemma A.2 in Appendix A, the Fourier transform of $\psi_{1}\left(t,\mu\right)$ in (3.6) in the argument $\mu$ is undefined for each fixed, sufficiently large $t$, and we cannot apply Fourier transform to $\psi_{1}\left(t,\mu\right)$ in the argument $\mu$ to obtain the Fourier transform of $K_{1}$ (and then invert the Fourier transform of $K_{1}$ to obtain $K_{1}$). This is the key difference between constructing $K$ for a one-sided null and those for a point null or bounded null, and requires additional innovation. Instead, finding $K_{1}$ for a one-sided null employs a different method, and $K_{1}$ is the real part of

[TABLE]

see details in the proof of Theorem 4.4. If we set $\psi\left(t,\mu\right)=2^{-1}-\psi_{1}\left(t,\mu\right)-2^{-1}\psi_{1,0}\left(t,\mu;0\right)$, then $\lim_{t\rightarrow\infty}\psi\left(t,\mu\right)=1_{\left(-\infty,0\right)}\left(\mu\right)$ due to (3.3). So, if we set $K\left(t,x\right)=2^{-1}-K_{1}\left(t,x\right)-2^{-1}K_{1,0}\left(t,x;0\right)$, then the pair $\left(\psi,K\right)$ solves the Lebesgue-Stieltjes equation (2.2).

In Figure 1, we provide visualizations of the pair $\left(\psi,K\right)$ for the point null $\Theta_{0}=\left\{0\right\}$, bounded null $\Theta_{0}=\left(-1,2\right)$, and one-sided null $\Theta_{0}=\left(-\infty,0\right)$, respectively, when each $F_{\mu}$ is the CDF of $X_{\mu}\sim\mathcal{N}_{1}\left(\mu,1\right)$ and $\omega\left(s\right)=\left(1-\left|s\right|\right)1_{\left[-1,1\right]}\left(s\right)$, i.e., the triangular density on $\left[-1,1\right]$ is used. Details on how we numerically compute $\left(\psi,K\right)$ are given in the beginning of Appendix G. We see that $K$ is an oscillator in each construction.

4 Constructions for Type I location-shift families

Recall $\mathcal{F}=\left\{F_{\mu}:\mu\in U\right\}$ or $\mathcal{F}=\left\{G_{\theta}:\theta\in\Theta\right\}$. We will refer to as “Construction I” the construction of estimators of $\pi_{1,m}$ for the bounded null $\Theta_{0}=\left(a,b\right)\cap U$ for fixed, finite $a,b\in U$ with $a<b$, and as “Construction II” the construction of estimators of $\pi_{1,m}$ for the one-sided null $\Theta_{0}=\left(-\infty,b\right)\cap U$ for a fixed, finite $b\in U$. Both constructions utilize Theorem 2.1, Dirichlet integrals, and their integral representations provided in Appendix A. In this section, we will provide the constructions when $\mathcal{F}$ is a Type I location-shift family. By default, each $F_{\mu}\in\mathcal{F}$ is uniquely determined by $\mu$. So, all $F_{\mu},\mu\in U$ have the same scale parameter, if any, when $\mathcal{F}$ is a location-shift family.

4.1 The case of a bounded null

Construction I for the bounded null for a Type I location-shift family is provide by

Theorem 4.1**.**

Assume $\mathcal{F}$ is a Type I location-shift family. Set

[TABLE]

Then

[TABLE]

and the desired $\left(\psi,K\right)$ is

[TABLE]

Note $\lim_{t\to\infty}\psi\left(t,\mu\right)=1_{\left(a,b\right)}\left(\mu\right)$ for $\psi\left(t,\mu\right)$ in (4.3). Define

[TABLE]

and

[TABLE]

Then $g$ measures the average reciprocal modulus of the CF $\hat{F}_{\mu}$ of $F_{\mu}$ on $\left[-1,1\right]$. As already shown by Chen (2019), $g$ plays a critical role in bounding the oscillation of $e_{m}\left(t\right)$ for the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$, and a larger $g$ usually gives a smaller maximal speed of convergence for $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ to achieve consistency. In contrast, $u_{m}$ measures the minimal distance from $\mu_{j}$ to the boundary points $a$ and $b$ of the bounded null $\Theta_{0}$, and a suitable magnitude for $u_{m}$ is needed for the estimator induced by $K_{1,0}\left(t,x;a\right)$ and $K_{1,0}\left(t,x;b\right)$ in (4.3) to consistently estimate the proportion of $\mu_{j}$’s that are equal to $a$ or $b$; see Theorem 4.2 below and Theorems 2 and 3 of Chen (2019).

Theorem 4.2**.**

Suppose $\left\{z_{i}\right\}_{i=1}^{m}$ are independent whose CDFs are members of a Type I location-shift family. Then

[TABLE]

and with probability at least $1-4\exp\left(-2^{-1}\lambda^{2}\right)$

[TABLE]

Further, (2.6) holds if there are positive sequences $\lambda_{m}\rightarrow 0$ and $t_{m}\rightarrow\infty$ such that

[TABLE]

Theorem 4.2 implicitly classifies settings for the consistency of $\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)$ for a Type I location-shift family but does not provide details on them. It states that the variance of the error $e_{m}\left(t\right)$ of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is characterized by the average reciprocal modulus $g$ at the generating measure $F_{0}$ of the location-shift family $\mathcal{F}$, the supremum norm $\|\omega\|_{\infty}$ of the employed density $\omega$, the “size” $b-a$ of the bounded null, the number $m$ of hypotheses to test, and the magnitude of the tuning parameter $t$. Our next task is to explore the uniform consistency of $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ and give a refined statement to Theorem 4.2, for which the following definition is needed and quoted from Chen (2019):

Definition 4.1**.**

Given a family $\mathcal{F}$, the sequence of sets \mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}\right)\subseteq\mathbb{R}^{m}\times\mathbb{R}\for each $m\in\mathbb{N}$ is called a “uniform consistency class” for the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ if

[TABLE]

If (4.6) holds and the $t$-section of $\mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}\right)$ (that is a subset of $\mathbb{R}^{m}$ containing $\boldsymbol{\mu}$) does not converge to the empty set in $\mathbb{R}^{\mathbb{N}}$ as $m\rightarrow\infty$, then $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is said to be “uniformly consistent”.

In words, a uniform consistency class for an estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is the asymptotic setting (as $m\to\infty$) for the family $\mathcal{F}$ under which $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ at its maximal speed of convergence (that is indicated by $\sup\nolimits_{t\in\mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}\right)}$ in Definition 4.1) still maintains consistency uniformly over all settings of all $F_{\mu_{i}},i=1,\ldots,m$ (that is indicated by $\sup\nolimits_{\boldsymbol{\mu}\mathcal{\in Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}\right)}$ in Definition 4.1) and hence uniformly over all settings of the alternative proportion $\pi_{1,m}$ that are induced by such $F_{\mu_{i}},i=1,\ldots,m$. Define

[TABLE]

The set $\mathcal{B}_{m}\left(\rho\right)$ for a fixed $\rho$ allows $\lim_{m\rightarrow\infty}\max_{1\leq j\leq m}\left|\mu_{j}\right|=\infty$, is just the $L_{1}$-ball of radius $\rho$ for $\boldsymbol{\mu}$, and was used by Jin (2008) and others in various estimation or testing problems. For $\mu\in U$, let $X_{\left(\mu\right)}$ have CDF $F_{\mu}$ and $A_{\mu}$ be the variance of $\left|X_{\left(\mu\right)}\right|$ (if $A_{\mu}$ is defined). Recall $u_{m}$ defined by (4.5). We have the following result on the uniform consistency class of the estimator:

Theorem 4.3**.**

Assume that $\mathcal{F}$ is a Type I location-shift family for which $\int\left|x\right|^{2}dF_{\mu}\left(x\right)<\infty$ for each $\mu\in U$. If $\left\{z_{i}\right\}_{i=1}^{m}$ are independent, then a uniform consistency class for the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is

[TABLE]

for constants $q$, $\gamma$, $\vartheta$ and $\vartheta^{\prime}$, where $\gamma_{m}=\gamma\ln m$, $R\left(\rho\right)=2\max_{\tau\in\left\{0,a,b\right\}}\mathbb{E}\left(\left|X_{\left(\tau\right)}\right|\right)+2\rho+2\max\left\{\left|a\right|,\left|b\right|\right\}$ and

[TABLE]

Moreover, for all sufficiently large $m$, with probability at least $1-o\left(1\right)$

[TABLE]

Theorem 4.3 is a very general and informative result, and is applicable whenever each CDF $F_{\mu}$ of a Type I location-shift family has finite variance. It characterizes via (4.9) the performance of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ at its maximal speed $\tau_{m}$ of convergence uniformly over the set $\mathcal{B}_{m}\left(\rho\right)$ for $\boldsymbol{\mu}$, where $\lim_{m\to\infty}\rho=\infty$ is allowed rather than $\rho$ being fixed. Even though the constant $q$ in Theorem 4.3 is not explicitly given, it can be easily determined (together with $\tau_{m}$), so that the upper bound in (4.9) converges to [math]. With this, from $t\Upsilon\left(0,q,\tau_{m},\gamma_{m}\right)=o\left(\pi_{1,m}\right)$ in (4.8), we can also see the class of alternative proportion $\pi_{1,m}$ for which $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is uniformly consistent at its maximal speed $\tau_{m}$.

It is informative to compare the set $\mathcal{Q}_{m}$ based on (4.8) and the uniform consistency class $\mathcal{\tilde{Q}}_{m}$ for a point null hypothesis given by Theorem 3 of Chen (2019), both for Gaussian random variables $\left\{z_{i}\right\}_{i=1}^{m}$ each with mean $\mu_{i}$ and standard deviation $1$. For this scenario, setting $t=\sqrt{2\gamma\ln m}$ yields

[TABLE]

and

[TABLE]

So, the speeds of convergence of corresponding estimators have the same order as $\sqrt{\ln m}$, even though for $\mathcal{\tilde{Q}}_{m}$ the maximal speed is $\sqrt{\ln m}$ which is achieved when $\liminf_{m\rightarrow\infty}\pi_{1,m}>0$. However, since $t^{-1}\left(1+u_{m}^{-1}\right)=o\left(\pi_{1,m}\right)$ for $\mathcal{Q}_{m}$ as demanded by the speed of convergence of the Oracle $\Lambda_{m}\left(\boldsymbol{\mu}\right)$, the sparsest $\pi_{1,m}$ contained in $\mathcal{Q}_{m}$ is usually larger in order than $t^{-1}$. In contrast, for $\mathcal{\tilde{Q}}_{m}$ the speed of convergence of the corresponding Oracle $\Lambda_{m}\left(\boldsymbol{\mu}\right)$ does not depend on $\pi_{1,m}$ but depends only on $u_{m}$, and thus the sparsest $\pi_{1,m}$ contained in $\mathcal{\tilde{Q}}_{m}$ can be of order arbitrarily close to (even though not equal to) $m^{-0.5}$.

For two Type I location-shift families $\mathcal{F}_{1}$ and $\mathcal{F}_{2}$, their corresponding uniform consistency classes $\mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}_{1}\right)$ and $\mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}_{2}\right)$ of the form (4.8), made to have the same constants $q,\gamma$, $\vartheta^{\prime}$ and $\vartheta$ and the same $\tau_{m}$, $\gamma_{m}$ and $u_{m}$, satisfy $\mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}_{1}\right)\subseteq\mathcal{Q}_{m}\left(\boldsymbol{\mu},t;\mathcal{F}_{2}\right)$ when $\mathcal{F}_{1}\succeq\mathcal{F}_{2}$, where the ordering $\succeq$ means that $r_{1,\mu}\left(t\right)\geq r_{2,\mu}\left(t\right)$ for all $\mu\in U$ and $t\in\mathbb{R}$ and $r_{i,\mu}$ is the modulus of the CF $\hat{F}_{i,\mu}$ of an $F_{i,\mu}\in\mathcal{F}_{i}$ for $i\in\left\{1,2\right\}$. This was also shown by the discussion below Theorem 3 in Chen (2019) for the setting of a point null hypothesis. Roughly speaking, the larger the moduli of the CF’s are, the larger the sparsest alternative proportion an estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is able to consistently estimate, and the more likely it has a slower maximal speed of convergence to achieve consistency.

4.2 The case of a one-sided null

When $\mathcal{F}$ is a location-shift family, it suffices to set $b=0$ for the one-sided null $\Theta_{0}=\left(-\infty,b\right)$. Construction II for the one-sided null for a Type I location-shift family is provide by

Theorem 4.4**.**

Suppose $\Theta_{0}=\left(-\infty,0\right)$. Assume $\mathcal{F}$ is a Type I location-shift family such that $F_{0}$ is differentiable, $\int\left|x\right|dF_{\mu}\left(x\right)<\infty$ for each $\mu\in U$ and

[TABLE]

Set

[TABLE]

Then

[TABLE]

and the desired $\left(\psi,K\right)$ is

[TABLE]

Set $K_{1}\left(t,x\right)=\Re\left\{K_{1}^{\dagger}\left(t,x\right)\right\}$. If $\partial_{t}\left\{1/r_{0}\left(t\right)\right\}$ is odd in $t$, then

[TABLE]

whereas if $\partial_{t}\left\{1/r_{0}\left(t\right)\right\}$ is even in $t$, then

[TABLE]

Note that $\lim_{t\to\infty}\psi\left(t,\mu\right)=1_{\left(-\infty,0\right)}\left(\mu\right)$ for $\psi\left(t,\mu\right)$ in (4.12). Also note that if a random variable has finite variance, then it must have finite first-order absolute moment. So, the condition “$\int\left|x\right|dF_{\mu}\left(x\right)<\infty$ for each $\mu\in U$” in Theorem 4.4 is weaker than the condition “$\int\left|x\right|^{2}dF_{\mu}\left(x\right)<\infty$ for each $\mu\in U$” in Theorem 4.3. Theorem 4.4 cannot be applied to the Cauchy family since none of its members has finite first-order absolute moment. However, it is applicable to the Gaussian family (as given by the example below) and three other families given in Section 4.3 (and others that are not presented in this work).

Example 4.1**.**

Gaussian family $\mathcal{N}\left(\mu,\sigma^{2}\right)$ with mean $\mu$ and standard deviation $\sigma>0$, for which

[TABLE]

The CF of $f_{\mu}$ is $\hat{f}_{\mu}\left(t\right)=\exp\left(\iota t\mu\right)\exp\left(-2^{-1}t^{2}\sigma^{2}\right)$. So, $r_{\mu}^{-1}\left(t\right)=\exp\left(2^{-1}t^{2}\sigma^{2}\right)$ and $\hat{f}_{0}=r_{0}$. Further,

[TABLE]

and condition (4.10) is satisfied.

For the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$, the concentration property of the difference $e_{m}\left(t\right)$ (defined by (2.5)) depends critically on $\partial_{t}\left\{1/r_{0}\left(t\right)\right\}$, as can be inferred from the results in Section 4.1. However, properties of $\partial_{t}\left\{1/r_{0}\left(t\right)\right\}$ can be quite different for different distribution families (as illustrated by the examples in Section 4.3), and a general treatment on the oscillation of $e_{m}\left(t\right)$ can be very notationally cumbersome. So, we will focus on the Gaussian family and have a uniform consistency class of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ given by

Theorem 4.5**.**

Assume $\Theta_{0}=\left(-\infty,0\right)$. Suppose $\left\{z_{i}\right\}_{i=1}^{m}$ are independent Gaussian random variables with identical variance $\sigma^{2}>0$. Then

[TABLE]

where $D_{m}=\sigma^{2}+m^{-1}\sum_{i=1}^{m}\mu_{i}^{2}$. Further, for $t>0$ and a fixed $\lambda>0$

[TABLE]

with probability at least $q_{m}^{\ast}\left(\lambda\right)=1-4\exp\left(-2^{-1}\lambda^{2}m\right)-m^{-1}\lambda^{-2}D_{m}$, and a uniform consistency class for $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is

[TABLE]

where $\tilde{u}_{m}=\min_{\left\{j:\mu_{j}\neq 0\right\}}\left|\mu_{j}\right|$.

We remarked that $\tilde{u}_{m}$ in Theorem 4.5 should be set as $\tilde{u}_{m}=\min_{\left\{j:\mu_{j}\neq 0\right\}}\left|\mu_{j}-b\right|$ when $b\neq 0$ in the one-sided null $\Theta_{0}=\left(-\infty,b\right)$. Similar to the $u_{m}$ in (4.5), a suitable magnitude for $\tilde{u}_{m}$ is needed for the estimator induced by $K_{1,0}\left(t,x;0\right)$ in (4.12) to consistently estimate the proportion of $\mu_{j}$’s that are equal to $b$; see Theorems 2 and 3 of Chen (2019). Similar to the bounds given by Theorem 4.2 and Theorem 4.3, Theorem 4.5 states that the variance of the error $e_{m}\left(t\right)$ of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ is characterized by the average reciprocal modulus $g$ at the generating measure $F_{0}$ of the location-shift family $\mathcal{F}$, the supremum norm $\|\omega\|_{\infty}$ of the employed density $\omega$, the “effective size” $m^{-1}\sum_{i=1}^{m}\mu_{i}^{2}$ of the one-sided null, the scale parameter $\sigma$ of the Gaussian family, the number $m$ of hypotheses to test, and the magnitude of the tuning parameter $t$. Further, the lager $g$ or $m^{-1}\sum_{i=1}^{m}\mu_{i}^{2}$ is, the more likely the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ will have a slower maximal speed of convergence to achieve consistency.

4.3 Additional examples of Type I location-shift families

We provide four additional Type I location-shift families, all of which were discussed in Section 3.1 of Chen (2019). Theorem 4.4 applies to each of the Laplace, Logistic and Hyperbolic Secant families (given below), and for each such family, a uniform consistency class for the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ can be obtained for the case of a bounded null using the same techniques in Section 4.1 and for the case of a one-sided null using the same techniques in Section 4.2, respectively. However, we will not pursue this here.

Example 4.2**.**

Laplace family $\mathsf{Laplace}\left(\mu,2\sigma^{2}\right)$ with mean $\mu$ and standard deviation $\sqrt{2}\sigma>0$ for which

[TABLE]

and the CF of $f_{\mu}$ is $\hat{f}_{\mu}\left(t\right)=\left(1+\sigma^{2}t^{2}\right)^{-1}\exp\left(\iota t\mu\right)$. So, $r_{\mu}^{-1}\left(t\right)=1+\sigma^{2}t^{2}$ and $\hat{f}_{0}=r_{0}$. Further,

[TABLE]

condition (4.10) is satisfied, and Theorem 4.4 applies to this example.

Example 4.3**.**

Logistic family $\mathsf{Logistic}\left(\mu,\sigma\right)$ with mean $\mu$ and scale parameter $\sigma>0$, for which

[TABLE]

and the CF of $f_{\mu}$ is $\hat{f}_{\mu}\left(t\right)=\frac{\pi\sigma t}{\sinh\left(\pi\sigma t\right)}\exp\left(\iota t\mu\right)$. So, $r_{\mu}^{-1}\left(t\right)=\frac{\sinh\left(\pi\sigma t\right)}{\pi\sigma t}$ and $\hat{f}_{0}=r_{0}$. Further,

[TABLE]

condition (4.10) is satisfied, and Theorem 4.4 applies to this example.

Example 4.4**.**

Cauchy family $\mathsf{Cauchy}\left(\mu,\sigma\right)$ with median $\mu$ and scale parameter $\sigma>0$, for which

[TABLE]

and the CF of $f_{\mu}$ is $\hat{f}_{\mu}\left(t\right)=\exp\left(-\sigma\left|t\right|\right)\exp\left(\iota t\mu\right)$. Since $\int\left|x\right|dF_{\mu}\left(x\right)=\infty$, Theorem 4.4 cannot be applied to the case of one-sided null.

Example 4.5**.**

Hyperbolic Secant family $\mathsf{HSecant}\left(\mu,\sigma\right)$ with mean $\mu$ and scale parameter $\sigma>0$, for which

[TABLE]

see, e.g., Chapter 1 of Fischer (2014). The identity

[TABLE]

implies $\hat{F}_{\mu}\left(t\right)=\sigma^{-1}\exp\left(-\iota t\mu\sigma^{-1}\right){\operatorname{sech}}\left(t\sigma^{-1}\right)$. So, $r_{\mu}^{-1}\left(t\right)=\sigma\cosh\left(t\sigma^{-1}\right)$ and $\hat{F}_{0}=r_{0}$. Further,

[TABLE]

condition (4.10) is satisfied, and Theorem 4.4 applies to this example.

5 Constructions for Gamma family

In this section, we present Construction I and Construction II when $\mathcal{F}$ is the Gamma family, for which the bounded null $\Theta_{0}=\left(a,b\right)\cap U$ and the one-sided null $\Theta_{0}=\left(-\infty,b\right)\cap U$ has to be convex sets. We write $\theta\left(\mu\right)$ as $\theta_{\mu}$, so that $\theta_{0}=\theta\left(0\right)$, $\theta_{a}=\theta\left(a\right)$ and $\theta_{b}=\theta\left(b\right)$, and $\mu_{0}=\mu\left(\theta_{0}\right)$, $a=\mu\left(\theta_{a}\right)$ and $b=\mu\left(\theta_{b}\right)$. Recall from Section 2.3 that $\mu\left(\theta\right)=\xi\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{1}$ for $\theta\in\Theta$ with $\xi\left(\theta\right)=\left(1-\theta\right)^{-1}$, $\zeta\left(\theta\right)\equiv\zeta_{0}=1$ and $\tilde{a}_{1}=\sigma$.

5.1 The case of a bounded null

Construction I for the bounded null for the Gamma family is provided below:

Theorem 5.1**.**

When $\mathcal{F}$ is the Gamma family, set

[TABLE]

Then

[TABLE]

and the desired $\left(\psi,K\right)$ is

[TABLE]

Note that the limit as $t\to\infty$ of $\psi_{1}\left(t,\theta\right)$ in (5.2) is the Dirichlet integral in (3.2) with $\mu$ as $\mu\left(\theta\right)$, which implies $\lim_{t\to\infty}\psi\left(t,\theta\right)=1_{\left(a,b\right)}\left(\mu\right)$ for $\psi\left(t,\theta\right)$ in (5.3). For the uniform consistency of the proposed estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$, set $\left\|1-\boldsymbol{\theta}\right\|_{\infty}=\max_{1\leq i\leq m}\left\{1-\theta_{i}\right\}$,

[TABLE]

Then we have

Theorem 5.2**.**

Suppose $\left\{z_{j}\right\}_{j=1}^{m}$ are independent Gamma random variables with parameters $\left\{\left(\theta_{i},\sigma\right)\right\}_{i=1}^{m}$. Then, when $t$ is positive and sufficiently large,

[TABLE]

Further, when $\sigma\geq 11/4$

[TABLE]

for each $\gamma\in\left(0,1\right]$ is a uniform consistency class, and when $\sigma\leq 3/4$, a uniform consistency class

[TABLE]

for each $\gamma\in\left(0,1\right)$.

Theorem 5.2 reveals that the uniform consistency class of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ changes with the scale parameter $\sigma$ of the Gamma family, which is similar to that for the estimator in the setting of a Type I location-shift family that has a scale parameter. However, the expression (5.6) states that when $\sigma\leq 3/4$, the maximal speed of convergence of $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ that maintains uniform consistency is independent of $\sigma$, which is similar to that of the estimator in the setting of a Gaussian family whose scale parameter (i.e., the standard deviation) is upper bounded by $1$. Since $\theta<1$ for the Gamma family, $u_{3,m}$ measures how close the parameter $\theta_{i}$ of a $G_{\theta_{i}}$ is to the singularity parameter $1$ for which a Gamma density is undefined, and it is sensible to assume $\liminf_{m\rightarrow\infty}u_{3,m}>0$. On the other hand, $\sigma\xi\left(\theta\right)=\mu\left(\theta\right)$ for all $\theta\in\Theta$. So, $\tilde{u}_{3,m}$ measures the minimal difference between the means $\mu\left(\theta_{i}\right)$ of $G_{\theta_{i}}$ for $\theta_{i}\notin\left\{\theta_{a},\theta_{b}\right\}$ and the means $\mu\left(\theta_{a}\right)$ and $\mu\left(\theta_{b}\right)$, and $\tilde{u}_{3,m}$ cannot be too small relative to $t$ as $t\rightarrow\infty$ in order for the estimator induced by $K_{3,0}\left(t,x;\theta_{a}\right)$ and $K_{3,0}\left(t,x;\theta_{b}\right)$ (already given by Theorem 2.1) in (5.3) to consistently estimate the proportions of means that are equal to $\mu\left(\theta_{a}\right)$ or $\mu\left(\theta_{b}\right)$; see Theorem 9 of Chen (2019). Finally, $\left\|1-\boldsymbol{\theta}\right\|_{\infty}$ measures the maximal range of $\left\{\theta_{i}\right\}_{i=1}^{m}$ from $1$, and when $\sigma\geq 11/4$, the larger $\left\|1-\boldsymbol{\theta}\right\|_{\infty}$ is, the more likely the estimator will have a slower maximal speed of convergence to achieve consistency.

5.2 The case of a one-sided null

We present Construction II for the one-sided null for the Gamma family as

Theorem 5.3**.**

When $\mathcal{F}$ is the Gamma family, set

[TABLE]

Then

[TABLE]

and the desired $\left(\psi,K\right)$ is

[TABLE]

Note that the limit as $t\to\infty$ of $\psi_{1}\left(t,\theta\right)$ in (5.7) is the Dirichlet integral in (3.3) with $\mu$ as $\mu\left(\theta\right)$, which implies $\lim_{t\to\infty}\psi\left(t,\theta\right)=1_{\left(-\infty,b\right)}\left(\mu\right)$ for $\psi\left(t,\theta\right)$ in (5.8). Recall u_{3,m}\in (5.4) and define $\check{u}_{3,m}=\min_{\left\{j:\theta_{j}\neq\theta_{b}\right\}}\left|\xi\left(\theta_{b}\right)-\xi\left(\theta_{j}\right)\right|$. A suitable magnitude of $\check{u}_{3,m}$ is needed for the estimator induced by $K_{3,0}\left(t,x;\theta_{b}\right)$ (already given by Theorem 2.1) in (5.8) to consistently estimate the proportion of $\mu\left(\theta_{i}\right)$’s that are equal to $\mu\left(\theta_{b}\right)$; see Theorem 9 of Chen (2019). We have the uniform consistency of the proposed estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ as

Theorem 5.4**.**

Assume $\left\{z_{j}\right\}_{j=1}^{m}$ are independent Gamma random variables with parameters $\left\{\left(\theta_{i},\sigma\right)\right\}_{i=1}^{m}$. Then, when $t$ is positive and sufficiently large,

[TABLE]

where

[TABLE]

Further, when $\sigma\geq 11/4$

[TABLE]

is a uniform consistency class for each $\gamma\in(0,1]$, and when $\sigma\leq 2^{-1}\sqrt{2}$

[TABLE]

is a uniform consistency class for each $\gamma\in\left(0,1\right)$.

In terms of the uniform consistency class and maximal speed of convergence of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$, Theorem 5.4 reveals similar things as does Theorem 5.2. In particular, the maximal speed of convergence $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ that maintains uniform consistency in the setting of one-side null does not depends on the scale parameter $\sigma$ when $\sigma\leq 2^{-1}\sqrt{2}$. When $\theta=1/2$ and $\sigma$ is a positive, even integer or when $\sigma=1$, the corresponding Gamma distribution becomes a central Chi-square distribution with degrees of freedom $2^{-1}\sigma$ or the exponential distribution with mean $(1-\theta)^{-1}$. So, Theorem 5.2 and Theorem 5.4 can be applied to proportion estimation for central Chi-square and exponential random variables.

We remark that it is much harder to obtain the bounds and uniform consistency classes given by Theorem 5.2 and Theorem 5.4 for the Gamma family than those given by Theorem 4.3 and Theorem 4.5 respectively for a Type I location-shift family and Gaussian family. This is because the matching function $K$ for the Gamma family uses a power series of a random variable in this family and we have to use asymptotics of Gamma and Bessel functions to bound (relatively tightly) the moments of a Gamma distribution in order to obtain concentration inequalities on the error term $e_{m}(t)$ of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$.

6 Extension of constructions for a bounded null

We extend the previous constructions of the setting of a bounded null to the setting of estimating the “induced proportion of true null hypotheses” , i.e., to estimate

[TABLE]

for a suitable function $\phi$. In this setting, $\check{\pi}_{0,m}\in\left[0,1\right]$ does not necessarily hold. For example, $\phi(x)=|x|^{p}$ for some $p>0$ gives $\check{\pi}_{0,m}$ as the “average $l^{p}$-norm” as a measure of sparsity for the vector $\boldsymbol{\mu}=\left(\mu_{1},...,\mu_{m}\right)^{\top}$, and $\phi(x;c_{\ast})=\min\{|x|^{p},c_{\ast}^{p}\}$ for some $p,c_{\ast}>0$ gives $\check{\pi}_{0,m}$ as the proportion of “signals” exceeding a threshold $c_{\ast}$, which were discussed by Jin (2008) and covered by his constructions. Such $\check{\pi}_{0,m}$ has applications in genomics and signal processing as mentioned in Section 1.

To formulate the extension, we need a crucial auxiliary result that is different than those provided in Section A.3. For a $\phi\in L^{1}\left(\left[a,b\right]\right)$ with finite $a$ and $b$ such that $a<b$, define

[TABLE]

Then we have

Lemma 6.1**.**

If $\phi\in L^{1}\left(\left[a,b\right]\right)$, then setting $\hat{\phi}\left(s\right)=\int_{a}^{b}\phi\left(y\right)\exp\left(-\iota ys\right)dy$ gives

[TABLE]

On the other hand, if $\phi$ is continuous and of bounded variation on $\left[a,b\right]$, then

[TABLE]

and

[TABLE]

Lemma 6.1 gives the relationship between the Fourier transform of $\phi$ and how to invert its Fourier transform, and will be used to derive Theorem 6.1 below. Note that (6.2) is almost the inverse of the Fourier transform of $\phi$. We caution that (6.3) does not necessarily hold when $\phi$ is only continuous, as can be seen from the examples in Chapter VIII of Zygmund (1959). Note also that (6.4) gives the speed of convergence of $\mathcal{D}_{\phi}\left(t,\mu;a,b\right)$ and helps determine the speed of convergence of the estimators to be constructed below:

Theorem 6.1**.**

Let $\phi$ be continuous and of bounded variation on $\left[a,b\right]$. Assume $\mathcal{F}$ is a Type I location-shift family and set

[TABLE]

Then

[TABLE]

and the desired $\left(\psi,K\right)$ for estimating $\check{\pi}_{0,m}$ is

[TABLE]

In contrast, assume $\mathcal{F}$ is the Gamma family, and set

[TABLE]

Then

[TABLE]

and the desired $\left(\psi,K\right)$ for estimating $\check{\pi}_{0,m}$ is

[TABLE]

Note that $\lim_{t\to\infty}\psi\left(t,\mu\right)=1_{\left(a,b\right)}\left(\mu\right)$ for $\psi\left(t,\mu\right)$ in both (6.7) and (6.10). The constructions in Theorem 6.1 can be easily modified to estimate any linear function of $\check{\pi}_{0,m}$, which will not be discussed here. In particular, if we set $a=-b$ with $b>0$ and take $K_{1}$ in (6.5) and $\psi_{1}$ in (6.6), then the construction $\left(\psi_{1},K_{1}\right)$ reduces to those for Gaussian family in Section 6 of Jin (2008). Moreover, when $\phi$ is the identity function on $\left[a,b\right]$, (6.5) and (6.8) respectively reduce to (4.1) and (5.1).

Define

[TABLE]

with $K$ in (6.7) or (6.10) and set $e_{m}\left(t\right)=\hat{\varphi}_{m}\left(t,\mathbf{z}\right)-\varphi_{m}\left(t,\boldsymbol{\mu}\right)$. Then $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ defined by (6.11) estimates $\check{\pi}_{0,m}$ defined by (6.1). Consistency of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ given by (6.11) can be obtained for independent $\left\{z_{i}\right\}_{i=1}^{m}$ via almost identical arguments as those for the proofs of Theorem 4.2, Theorem 4.3, and Theorem 5.2; see Theorem 6.2 and Corollary 6.1 below. For the rest of this section, we assume that $\phi$ is continuous and of bounded variation on $\left[a,b\right]$. Recall u_{m}\in (4.5) and $u_{3,m}$ and $\tilde{u}_{3,m}$ in (5.4). We have the uniform consistency of $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ as

Theorem 6.2**.**

Consider the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ in (6.11) and assume $\left\{z_{i}\right\}_{i=1}^{m}$ are independent whose CDFs are members of $\mathcal{F}$. If $\mathcal{F}$ is a Type I location-shift family, then

[TABLE]

and with probability at least $1-4\exp\left(-2^{-1}\lambda^{2}\right)$

[TABLE]

Further, $\Pr\left\{\left|\check{\pi}_{1,m}^{-1}\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)-1\right|\rightarrow 0\right\}\rightarrow 1\text{\quad as \quad}m\rightarrow\infty$ for positive sequences $\lambda_{m}\rightarrow 0$ and $t_{m}\rightarrow\infty$ such that

[TABLE]

On the other hand, if $\mathcal{F}$ is the Gamma family, then for all positive and large $t$

[TABLE]

and (5.5) is a uniform consistency class for each $\gamma\in\left(0,1\right]$ when $\sigma\geq 11/4$, and (5.6) a uniform consistency class for each $\gamma\in\left(0,1\right)$ when $\sigma\leq 3/4$, after replacing $\pi_{1,m}$ (5.5) and (5.6) by $\check{\pi}_{0,m}$.

The uniform consistency classes of the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ in (6.11) for a Type I location-shift family can be obtained by applying some algebra to the bounds provided by (6.12) in Theorem 6.2. However, sharper uniform consistency classes can be obtained for a specific Type I location-shift family. In particular, from Theorem 6.2, we have a uniform consistency class for the estimator $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ in (6.11) for the Gaussian family:

Corollary 6.1**.**

Assume $\left\{z_{i}\right\}_{i=1}^{m}$ are independent Gaussian random variables identical variances $\sigma^{2}>0$. Then a uniform consistency class for $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)$ in (6.11) is

[TABLE]

On the other hand, for the estimator $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$ with $K_{1}$ in (6.5) that estimates

[TABLE]

a uniform consistency class is

[TABLE]

The “uniform consistency class” in Corollary 6.1 bears the meaning of Definition 4.1 but with $\check{\pi}_{0,m}$ or $\tilde{\pi}_{0,m}$ in place of $\pi_{1,m}$. The second assertion of Corollary 6.1 on estimating $\tilde{\pi}_{0,m}$ complements and strengthens much Theorem 13 of Jin (2008), since the latter work in our notations requires $\phi$ to be absolutely continuous, deals with the case $\left[a,b\right]$ being a symmetric interval, and only shows $\sup_{\mathcal{\tilde{B}}_{m}\left(\rho\right)}\left|\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)-\tilde{\pi}_{0,m}\right|=o\left(1\right)$ for a subset $\mathcal{\tilde{B}}_{m}\left(\rho\right)$ of $\mathcal{B}_{m}\left(\rho\right)$ (that is defined by (4.7) and allows $\rho\to\infty$).

7 Discussion

For multiple testing a bounded or one-sided null on the means or medians of random variables whose CDFs are members of a Type I location-shift family or the Gamma family, we have constructed uniformly consistent estimators of the corresponding proportion of false null hypotheses via solutions to Lebesgue-Stieltjes integral equations. The strategy proposed in the Discussion section of Chen (2019) or that in Section 2.3 of Jin (2008), i.e., choosing a speed of convergence $t_{m}=\sqrt{\gamma\ln m}$ with some $\gamma\in\left(0,1\right)$ that controls the variance of the error term $e_{m}(t_{m})$ of a proportion estimator for a finite $m$, can be used to adaptively determine the speed of convergence (and hence the tuning parameter $\gamma$) for the proposed estimators. These estimators can be used to develop adaptive versions of the FDR procedure of Chen (2020), the “BH” procedure of Benjamini and Yekutieli (2001) under the conditions of their Theorem 5.2, the FDR and FNR procedures of Sarkar (2006), or any other conservative FDR or FNR procedure that is applicable to multiple testing composite null hypotheses.

Our settings and results can be extended in several aspects as follows. First, the constructions and uniform consistency of the proportion estimators provided here can be easily extended to the setting where the null parameter set belongs to the algebra generated by bounded, one-sided and point nulls. Here the term “algebra” refers to the family of sets generated by applying any finite combination of set union, intersection or complement to these three types of nulls. Second, the speed of convergence and uniform consistency class for Construction I and Construction II can be obtained for each non-Gaussian Type I location-shift family given in Section 4.3 to which our theory applies. Third, it is possible to establish the uniform consistency of the proportion estimators in Section 5 and Section 6 for weakly dependent random variables that are bivariate Gaussian via their associated Hermite polynomials or bivariate Gamma via their associated Laguerre polynomials. Fourth, for the settings of one-sided and bounded nulls and suitable functionals of the bounded null, respectively, following the principles in Section 3 of Jin (2008), all constructions we have provided can be applied to consistently estimate the mixing proportions for two-component mixture models at least one of whose components follows a Gaussian or Gamma distribution.

Our proportion estimators may lead to consistent estimators of or tests on the “sparsity level” of regression coefficients in high-dimensional, sparse Gaussian linear models, as explained below. Consider the model $\mathbf{y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}$, where $\mathbf{y}\in\mathbb{R}^{n}$ is the vector of observed response, $\mathbf{X}\in\mathbb{R}^{n\times p}$ a known design matrix, $\boldsymbol{\beta}\in\mathbb{R}^{p}$ the vector of unknown coefficients, and $\boldsymbol{\varepsilon}\sim\mathcal{N}_{n}\left(0,\boldsymbol{\Sigma}\right)$ the vector of random errors. Model selection via penalized least squares to consistently estimate $\boldsymbol{\beta}$ in the setting $p\gg n$ and $\min\left\{n,p\right\}\rightarrow\infty$ often assumes an order for the “sparsity level” $\varpi$, i.e., the number of nonzero entries of $\boldsymbol{\beta}$, relative to $n$ and $p$; see, e.g., Zhao and Yu (2006); Lv and Fan (2009); Zhang (2010); Su and Candés (2016); Javanmard and Montanari (2018). However, there does not seem to exist a consistent estimator of or test on $\varpi$ when $\mathbf{X}$ is not a diagonal matrix. If we set $\boldsymbol{\mu}=\mathbf{X}\boldsymbol{\beta}=\left(\mu_{1},\ldots,\mu_{n}\right)^{\intercal}$ and assume $\boldsymbol{\Sigma}=\sigma^{2}\boldsymbol{I}$ for some $\sigma^{2}>0$ where $\boldsymbol{I}$ is the identity matrix, then we can look at$\ddot{\pi}_{1,n}=n^{-1}\sum_{i=1}^{n}\tilde{\phi}\left(\mu_{i}\right)$ for suitable functions $\tilde{\phi}$ as test functions on properties of $\boldsymbol{\mu}$ and hence on $\boldsymbol{\beta}$, and consistent estimators of $\ddot{\pi}_{1,n}$ for various $\tilde{\phi}$ (which include $\phi$ discussed in Section 6) may lead to good lower or upper bounds on or consistent estimators of $\varpi$ for nontrivial $\mathbf{X}$ and $\boldsymbol{\beta}$.

Finally, our constructions essentially utilize suitable group structures of the domain of the parameters and arguments of the CDFs and then apply special transforms that are adapted to such group structures to obtain solutions to the Lebesgue-Stieltjes integral equation. This general principle is also applicable to probability distributions on Lie groups (which contains, e.g., the unit sphere a special case), and tools of harmonic analysis on Lie groups can be used to construction proportion estimators for these distributions. This has applications to modeling and analysis of data on Lie groups. We will report on all these in other articles.

Acknowledgements

This research was funded by the New Faculty Seed Grant provided by Washington State University. I am grateful to Prof. Gérard Letac for his constant guidance and feedback on an earlier version of the manuscript and to Prof. Jiashun Jin for his warm encouragements. In particular, the second part of the proof of Theorem A.2 was inspired by Prof. Gérard Letac’s suggestion on examining the ratio $\tilde{c}_{n+1}\left(\theta\right)/\tilde{c}_{n}\left(\theta\right)$ via the relationship $\frac{d}{d\theta}\int x^{n}G_{\theta}\left(dx\right)=\int x^{n+1}G_{\theta}\left(dx\right)$ for the Gamma family introduced in Section 2.3.

Appendix A Five auxiliary results and their proofs

We discuss in Section A.1 how to construct Type I location-shift families, show in Section A.2 that the only NEF with separable moments is the Gamma family, and provide in Section A.3 a lemma on Dirichlet integrals and two lemmas on the speed of convergence of a Fourier transform.

A.1 Constructing Type I location-shift families

Recall Definition 2.1 of Type I location-shift family in the main text. We first provide two examples that are not Type I location-shift families:

Example A.1**.**

Let

[TABLE]

where $\sigma=\sqrt{2/\pi}$. Then $\tilde{f}_{0}$ is a density function on $\mathbb{R}$. Further, $\int_{0}^{\infty}x\tilde{f}_{0}\left(x\right)dx=0.5$ and $\int_{-\infty}^{0}x\tilde{f}_{0}\left(x\right)dx=-1/\pi$. Let $X$ have density $\tilde{f}_{0}\left(x\right)$ and $Y=X-\left(0.5-1/\pi\right)$. Denote the density of $Y$ by $f_{0}$. Then $\mathbb{E}\left(Y\right)=0$ and $F_{\mu}\left(x\right)=\int_{-\infty}^{x}f_{0}\left(y-\mu\right)dy,\mu\in\mathbb{R}$ form a location-shift family $\mathcal{F}$. But this $\mathcal{F}$ is not a Type I location-shift family since $f_{0}$ is not an even function.

Example A.2**.**

Consider a Pólya-type characteristic function (CF) $\hat{F}\left(t\right)=\left\{1-\left|t\right|\right\}1_{\left\{\left|t\right|\leq 1\right\}}$ (see, e.g., Lukacs (1970)). Then the density function corresponding to $\hat{F}\left(t\right)$ is $f\left(x\right)=\pi^{-1}x^{-2}\left(1-\cos x\right)$ for $x\in\mathbb{R}$, and $F_{\mu}\left(x\right)=\int_{-\infty}^{x}f_{0}\left(y-\mu\right)dy,\mu\in\mathbb{R}$ form a location-shift family $\mathcal{F}$. However, $\hat{F}\left(t\right)=0$ for all $\left|t\right|\geq 1$. So, this $\mathcal{F}$ is not a Type I location-shift family.

Now let us describe two methods to construct Type I location-shift families, for which we need the following “Pólya’s condition” (see, e.g., Lukacs (1970)):

Lemma A.1** (G. Pólya).**

Let $h:\mathbb{R}\rightarrow\mathbb{R}$ be a continuous function. If $h$ is even, $h\left(0\right)=1$, $h\left(t\right)$ is convex for $t>0$, and $\lim_{t\rightarrow\infty}h\left(t\right)=0$, then $h$ is the CF of an absolutely continuous CDF.

A CF satisfying the conditions in Lemma A.1 is called a “Pólya-type CF”. Since the density function $f$ of an absolutely continuous CDF $F$ is even if and only if the CF $\hat{F}$ of $F$ is real, we see that a random variable $X$ whose CDF $F_{0}$ has a real CF $\hat{F}_{0}$ must have zero expection. So, Lemma A.1 tells us that, if we pick a Pólya-type CF $h_{0}$ that is nowhere [math] on $\mathbb{R}$ and is Lebesgue integrable on $\mathbb{R}$, then its Fourier inverse $\hat{h}_{0}$ generates a Type I location-shift family with members CDFs as $F_{\mu}\left(x\right)=\int_{-\infty}^{x}\hat{h}_{0}\left(y-\mu\right)dy,\mu\in\mathbb{R}$.

A second method to construct a Type I location-shift family is based on a key result of Tuck (2006), which can also be derived by the arguments in the proof of Theorem 4.3.1 of Lukacs (1970). This result is

Theorem A.1** (Tuck (2006)).**

If a function $f$ defined on $\mathbb{R}_{>0}$ is Lebesgue integrable on $\mathbb{R}_{>0}$ and strictly convex on $\mathbb{R}_{>0}$, then the function $g\left(t\right)=\int_{0}^{\infty}f\left(x\right)\cos\left(tx\right)dx$ is positive for all $t>0$.

Theorem A.1 says that a convex function has a positive “Fourier-cosine” transform, and it shows us a second way to construct Type I location-shift families, i.e., take an even density function $f_{0}$ on $\mathbb{R\ }$such that $f_{0}$ is strictly convex on $\mathbb{R}_{>0}$, then $F_{\mu}\left(x\right)=\int_{-\infty}^{x}f_{0}\left(y-\mu\right)dy,\mu\in\mathbb{R}$ form a Type I location-shift family.

Finally, the complex analytic techniques of Tuck (2006) can be used to construct Type I location-shift families from an even density function on $\mathbb{R}$ that is bell-shaped. These techniques mainly use analytic continuation, contour deformation, and residue calculus, which we will not detail here.

A.2 NEFs with separable moments

We provide a very brief review on natural exponential family (NEF), whose details can be found in Letac (1992). Let $\beta$ be a positive Radon measure on $\mathbb{R}$ that is not concentrated on one point. Let $L\left(\theta\right)=\int e^{x\theta}\beta\left(dx\right)$ for $\theta\in\mathbb{R}$ be its Laplace transform and $\Theta$ be the maximal open set containing $\theta$ such that $L\left(\theta\right)<\infty$. Suppose $\Theta$ is not empty and let $\kappa\left(\theta\right)=\ln L\left(\theta\right)$ be the cumulant function of $\beta$. Then

[TABLE]

forms an NEF with respect to the basis $\beta$. Note that $\Theta$ has a non-empty interior and is a convex set if it is not empty and that $L$ is analytic on the strip $A_{\Theta}=\left\{z\in\mathbb{C}:\Re\left(z\right)\in\Theta\right\}$.

The NEF $\mathcal{F}$ can be equivalently characterized by its mean domain and variance function. Specifically, the mean function $\mu:\Theta\rightarrow U$ with $U=\mu\left(\Theta\right)$ is given by $\mu\left(\theta\right)=\frac{d}{d\theta}\kappa\left(\theta\right)$, and the variance function is $V\left(\theta\right)=\frac{d^{2}}{d\theta^{2}}\kappa\left(\theta\right)$ and can be parametrized by $\mu$ as

[TABLE]

where $\theta=\theta\left(\mu\right)$ is the inverse function of $\mu$ and $F_{\mu}=G_{\theta\left(\mu\right)}$. Namely, $\mathcal{F}=\left\{F_{\mu}:\mu\in U\right\}$. The pair $\left(V,U\right)$ is called the variance function of $\mathcal{F}$, and it characterizes $\mathcal{F}$.

For the case of a point null when $\mathcal{F}$ is an NEF, Chen (2019) introduced the definition of “NEF with separable moment functions at a specific point”, showed that the Gamma family has such moment functions, and constructed a uniformly consistent proportion estimator for the Gamma family. In order to construct proportion estimators for a one-sided or bounded null or a functional of a bounded null when $\mathcal{F}$ is an NEF, we need a stronger definition as

Definition A.1**.**

For each $\theta\in\Theta$ and $G_{\theta}\in\mathcal{F}$ for an NEF $\mathcal{F}$, denote the moment sequence of $G_{\theta}$ by

[TABLE]

If there exist two functions $\zeta,\xi:\Theta\rightarrow\mathbb{R\ }$and a sequence of real numbers $\left\{\tilde{a}_{n}\right\}_{n\geq 0}$ that satisfy the following:

• •

$\xi$* is one-to-one, $\zeta\left(\theta\right)\neq 0$ for all $\theta\in U$, and $\zeta$ does not depend on any $n\in\mathbb{N}$,*

• •

$\tilde{c}_{n}\left(\theta\right)=\xi^{n}\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{n}$* for each $n\in\mathbb{N}$ and $\theta\in\Theta$,*

• •

$\Psi\left(t,\theta\right)=\sum_{n=0}^{\infty}\frac{t^{n}\xi^{n}\left(\theta\right)}{\tilde{a}_{n}n!}$* is absolutely convergent pointwise in $\left(t,\theta\right)\in\mathbb{R}\times\Theta$,*

then the moment sequence $\left\{\tilde{c}_{n}\left(\theta\right)\right\}_{n\geq 0}$ is called “separable” and $\mathcal{F}$ is said to have “separable moments”.

Definition A.1 requires that the moments of $G_{\theta}$ are separable in the sense of $\tilde{c}_{n}\left(\theta\right)=\xi^{n}\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{n}$ at all points in $\Theta$ rather than at just one point in $\Theta$. Note that $\mu\left(\theta\right)=\xi\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{1}$ for an NEF with separable moments. In the main text, we have restated that the Gamma family has separable moments. If there are NEFs with separable moments different than the Gamma family, then the proportion estimators of Chen (2019) and those proposed here will have wider applications. However, we have a somewhat disappointing result on how many NEFs with separable moments exist as

Theorem A.2**.**

The only NEF with separable moments is the Gamma family.

Proof.

By definition,

[TABLE]

By the assumption $\tilde{c}_{n}\left(\theta\right)=\xi^{n}\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{n}$ for $n\in\mathbb{N}$, we have $\mu\left(\theta\right)=\tilde{a}_{1}\xi\left(\theta\right)\zeta\left(\theta\right)$ and $\tilde{a}_{1}\neq 0$. However, $V\left(\theta\right)=\frac{d}{d\theta}\mu\left(\theta\right)$ and

[TABLE]

Since $V\left(\theta\right)>0$ and $\tilde{c}_{2}\left(\theta\right)>0$, we have $\zeta\left(\theta\right)\tilde{a}_{2}>0$ and from (A.1) we obtain

[TABLE]

Write $\tilde{h}\left(\mu\right)=\zeta\left(\theta\right)$ and $\tilde{V}\left(\mu\right)=V\left(\theta\right)$. Then

[TABLE]

and $\tilde{V}$ is real, positive, and analytic on the mean domain $U$.

From (A.3), we see that $\tilde{V}\left(\mu\right)$ is quadratic in $\mu$ if and only if $\zeta\left(\theta\right)$ is constant. However, setting $n=0$ for $\tilde{c}_{n}\left(\theta\right)$, we obtain

[TABLE]

So, $\tilde{a}_{0}\neq 0$ and $\zeta\equiv\tilde{a}_{0}^{-1}$. Thus, $\tilde{V}\left(\mu\right)$ is quadratic in $\mu$, and $\tilde{V}$ has a double root at zero and no other roots. By Letac and Mora (1990) and their Table 1, we conclude that the family $\mathcal{F}$ has to be the Gamma family. ∎

A.3 Dirichlet integral and Fourier transform

In order to construct the new proportion estimators and show their uniform consistency for the settings of a one-sided null, a bounded null and a suitable functional of a bounded null respectively, we introduce three auxiliary results that are of independent interest. First, we have the speed of convergence of Dirichlet integral as

Lemma A.2**.**

$\left|\int_{0}^{t}x^{-1}\sin xdx-2^{-1}\pi\right|\leq 2\pi t^{-1}$* for $t\geq 2$.*

Lemma A.2 implies the following identities (also referred to as “Dirichlet integral”) that will be used in the new constructions:

[TABLE]

and

[TABLE]

Further, we have the following identities that will be used in the integral representations of solutions to the Lebesgue-Stieltjes integral equation (2.2) in the constructions of the new proportion estimators:

Lemma A.3**.**

For any $a,b,\mu,t\in\mathbb{R}$ with $a<b$,

[TABLE]

On the other hand, for any $b,\mu,t\in\mathbb{R}$,

[TABLE]

Finally, we have the following to be used to derive the speed of the convergence of the Oracle proportion estimator $\Lambda_{m}\left(\boldsymbol{\mu}\right)$:

Lemma A.4**.**

Let $-\infty<a_{1}<b_{1}<\infty$. If $f:\left[a_{1},b_{1}\right]\rightarrow\mathbb{R}$ is of bounded variation, then

[TABLE]

Note that better bounds than (A.8) can be derived when $f$ has higher order derivatives (by adapting the techniques of Jackson (1920)) but are not our focus here and will not improve the speeds of convergence of the proportion estimators to be presented later.

The proofs of these three lemmas are given below in order.

A.3.1 Proof of Lemma A.2

Pick $\rho$ and $R$ such that $R>\rho>0$. Define the counterclockwise oriented contour $\tilde{C}=\bigcup\nolimits_{i=1}^{4}C_{i}$, where $C_{1}=\left\{Re^{\iota x}:0\leq x\leq\pi\right\}$, $C_{2}=\left\{\rho e^{\iota x}:\pi\geq x\geq 0\right\}$, $C_{3}=\left\{x:-R\leq x\leq-\rho\right\}$ and $C_{4}=\left\{x:\rho\leq x\leq R\right\}$. Let $f\left(z\right)=e^{\iota z}/z$ for $z\in\mathbb{C\setminus}\left\{0\right\}$. Then

[TABLE]

However,

[TABLE]

and

[TABLE]

and

[TABLE]

Since the ratio $\rho\left(1-\rho\right)^{-1}$ for $\rho\in\left(0,1\right)$ upper bounded by $2\rho$ when $\rho<2^{-1}$, and $\sin x\leq x$ for all $x\geq 0$, setting $\rho=R^{-1}$ gives

[TABLE]

for all $R\geq 2$. Thus, $\left|\int_{0}^{R}x^{-1}\sin xdx-2^{-1}\pi\right|\leq 2\pi R^{-1}$ for all $R\geq 2$.

A.3.2 Proof of Lemma A.3

By simple algebra, we have

[TABLE]

On the other hand,

[TABLE]

So, by a change of variable $\mu\mapsto\mu-b$ for the above identity, we have the claimed identity.

A.3.3 Proof of Lemma A.4

By Jordan’s decomposition theorem, $f=g_{1}-g_{2}$, where $g_{1}\left(x\right)=2^{-1}g_{0}\left(x\right)+2^{-1}f\left(x\right)$ and $g_{2}\left(x\right)=2^{-1}g_{0}\left(x\right)-2^{-1}f\left(x\right)$ are non-decreasing functions on $\left[a_{1},b_{1}\right]$ and $g_{0}\left(x\right)$ is the total variation of $f$ on $\left[a_{1},x\right]$ for $x\in\left[a_{1},b_{1}\right]$. So,

[TABLE]

For the first summand in (A.9), we can apply the second law of the mean to obtain

[TABLE]

for some $s_{0}\in\left[a_{1},b_{1}\right]$. So, $\left|I_{g_{1}}\right|\leq 2\left(b_{1}-a_{1}\right)\left\|g_{1}\right\|_{\infty}\left|t\right|^{-1}$ when $t\neq 0$, and

[TABLE]

when $t=0$. Applying the same arguments to the second summand $I_{g_{2}}$ in (A.9) yields $\left|I_{g_{2}}\right|\leq 2\left(b_{1}-a_{1}\right)\left\|g_{2}\right\|_{\infty}$ when $t=0$ and $\left|I_{g_{2}}\right|\leq 2\left(b_{1}-a_{1}\right)\left\|g_{2}\right\|_{\infty}\left|t\right|^{-1}$ when $t\neq 0$. However,

[TABLE]

Thus,

[TABLE]

Appendix B Proofs Related to Construction I for Type I location-shift family

B.1 Proof of Theorem 4.1

Pick any $\mu^{\prime}\in U$. Let $\tilde{g}\left(t;\mu^{\prime}\right)=\int_{a}^{b}\exp\left\{-\iota\left(y-\mu^{\prime}\right)t\right\}dy$ and

[TABLE]

Then,

[TABLE]

So, from Lemma A.3 we have

[TABLE]

Now set $\mu^{\prime}=0$. Since $\mathcal{F}$ is a Type I location-shift family, $\hat{F}_{0}\equiv r_{0}$ holds, and

[TABLE]

Namely, $Q_{1}\left(t,x;0\right)$ is exactly $K_{1}\left(t,x\right)$. Finally, we only need to capture the contributions of the end points $a$ and $b$ to estimating $\pi_{1,m}$. By Theorem 2.1, we only need to set $\left(K,\psi\right)$ as given by (4.3).

B.2 Proof of Theorem 4.2

Define $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$ and $\varphi_{1,m}\left(t,\boldsymbol{\mu}\right)=\mathbb{E}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}$. First, we study the concentration properties of $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)-\varphi_{1,m}\left(t,\boldsymbol{\mu}\right)$. Recall

[TABLE]

and $\psi_{1}\left(t,\mu\right)=\int K_{1}\left(t,x\right)dF_{\mu}\left(x\right)$. Set $w_{1,i}\left(v;y\right)=\cos\left\{v\left(z_{i}-y\right)\right\}$ for each $i$ and $v\in\mathbb{R}$ and $y\in\left[a,b\right]$. Define

[TABLE]

Then

[TABLE]

Since $\left|w_{1,i}\left(ts;y\right)\right|\leq 1$ uniformly in $\left(t,s,y,z_{i},i\right)$ and $\left\{z_{i}\right\}_{i=1}^{m}$ are independent, we have

[TABLE]

where we recall $g\left(t,\mu\right)=\int_{\left[-1,1\right]}1/r_{\mu}\left(ts\right)ds$.

Next we show the concentration property for $\hat{\varphi}_{m}\left(t,\mathbf{z}\right)-\varphi_{m}\left(t,\boldsymbol{\mu}\right)$. Recall

[TABLE]

and the functions $K_{1,0}$ and $\psi_{1,0}$ from Theorem 2.1. Define for $\tau\in\mathbb{R}$

[TABLE]

Let $\lambda>0$ be a constant. By Theorem 2 of Chen (2019), for any fixed $\tau\in U$, with probability at least $1-2\exp\left(-2^{-1}\lambda^{2}\right)$,

[TABLE]

and

[TABLE]

Combining (B.2) and (B.5), we have

[TABLE]

However, $\mathcal{F}$ is a location-shift family. So, $g\left(t,\mu\right)$ is independent of $\mu$, i.e., $g\left(t,0\right)=g\left(t,\mu\right)$ for all $\mu\in U$. So,

[TABLE]

Finally, we show the second assertion of theorem. By Hoeffding’s inequality of Hoeffding (1963),

[TABLE]

So, with probability $1-2\exp\left(-2^{-1}\lambda^{2}\right)$,

[TABLE]

and in view of (B.5), with probability at least $1-4\exp\left(-2^{-1}\lambda^{2}\right)$

[TABLE]

Consider the decomposition

[TABLE]

where

[TABLE]

Recall (B.4) and (B.6). Since $\left\|\omega\right\|_{\infty}<\infty$ and $m^{-1/2}\lambda_{m}t_{m}g\left(t_{m},0\right)=o\left(\pi_{1,m}\right)$, we have

[TABLE]

and it suffices to show $\pi_{1,m}^{-1}\tilde{r}_{0,m}=1+o\left(1\right)$. Recall

[TABLE]

from Theorem 2.1. Then, Lemma A.4 implies

[TABLE]

and

[TABLE]

where $u_{m}=\min_{\tau\in\left\{a,b\right\}}\min_{\left\{j:\mu_{j}\neq\tau\right\}}\left|\mu_{j}-\tau\right|$. This, together with Lemma A.2, implies

[TABLE]

Therefore, $t_{m}^{-1}\left(1+u_{m}^{-1}\right)=o\left(\pi_{1,m}\right)$ forces $\left|\pi_{1,m}^{-1}\tilde{r}_{0,m}-1\right|\rightarrow 0$. Since $\exp\left(-2^{-1}\lambda_{m}^{2}\right)\rightarrow 0$, we get

[TABLE]

B.3 Proof of Theorem 4.3

The strategy of proof is similar to that of Theorem 3 of Chen (2019). Recall $w_{1,i}\left(v;y\right)=\cos\left\{v\left(z_{i}-y\right)\right\}$ and $S_{1,m}\left(v;y\right)$ defined by (B.1). Let $\hat{s}_{1,m}\left(v;y\right)=m^{-1}\sum_{i=1}^{m}w_{1,i}\left(v;y\right)$ and $s_{1,m}\left(v;y\right)=\mathbb{E}\left\{\hat{s}_{1,m}\left(v;y\right)\right\}$. Then $S_{1,m}\left(v;y\right)=\hat{s}_{1,m}\left(v;y\right)-s_{1,m}\left(v;y\right)$. For the rest of the proof, we will first assume the existence of the positive constants $\gamma$, $q$, $\vartheta$ and the non-negative constant $\vartheta^{\prime}$ and then determine them. Let $\gamma_{m}=\gamma\ln m$ and define the closed interval $G_{m}=\left[0,\gamma_{m}\right]$. The rest of the proof is divided into three parts.

Part I: Recall that $X_{\left(\mu\right)}$ has CDF $F_{\mu}$ and that $A_{\mu}$ is the variance of $\left|X_{\left(\mu\right)}\right|$ for $\mu\in U$. Using almost identical arguments in Part I of the proof of Theorem 3 of Chen (2019), we can show the assertion: if

[TABLE]

where $R_{1,m}\left(\rho\right)=2\mathbb{E}\left\{\left|X_{\left(0\right)}\right|\right\}+2C_{a,b}+2\rho$ and $C_{a,b}=\max\left\{\left|a\right|,\left|b\right|\right\}$, then, for all large $m$,

[TABLE]

holds with probability at least $1-p_{m}\left(0,\vartheta,q,\gamma_{m}\right)$, where for $\mu\in U$

[TABLE]

To save space, we omit repeating them here.

Part II: to show the uniform bound on $\left|\hat{\varphi}_{m}\left(t,\mathbf{z}\right)-\varphi_{m}\left(t,\boldsymbol{\mu}\right)\right|$. Pick a positive sequence $\left\{\tau_{m}:m\geq 1\right\}$ such that $\tau_{m}\leq\gamma_{m}$ for all large $m$ and $\tau_{m}\rightarrow\infty$. Then, $S_{1,m}\left(ts;y\right)$ being even in $s\in\left[-1,1\right]$ and Part I together imply that, with probability at least $1-p_{m}\left(0,\vartheta,q,\gamma_{m}\right)$,

[TABLE]

for all sufficiently large $m$, where we recall for $\mu\in U$

[TABLE]

Note that $\Upsilon\left(\mu,q,\tau_{m},\gamma_{m}\right)=\Upsilon\left(0,q,\tau_{m},\gamma_{m}\right)$ for a location-shift family.

Recall (B.3) for the definitions of $\hat{\varphi}_{1,0,m}\left(t,\mathbf{z};\tau\right)$ and $\varphi_{1,0,m}\left(t,\boldsymbol{\mu};\tau\right)$. Since $\mathcal{F}$ is a Type I location-shift family, the argument $h_{\mu}\left(t\right)=\mu t$ and

[TABLE]

So, Part I of the proof of Theorem 3 of Chen (2019) yields the following assertion: for $\tau\in\left\{a,b\right\}$, if

[TABLE]

where $R_{0,m}\left(\rho,\tau\right)=2\mathbb{E}\left\{\left|X_{\left(\tau\right)}\right|\right\}+2\rho+2C_{a,b}$, then

[TABLE]

with probability at least $1-p_{m}\left(\tau,\vartheta,q,\gamma_{m}\right)$, where $p_{m}\left(\mu,\vartheta,q,\gamma_{m}\right)$ is defined by (B.9).

Define $p_{m}^{\ast}\left(\vartheta,q,\gamma_{m}\right)=3\max_{\tau\in\left\{0,a,b\right\}}p_{m}\left(\tau,\vartheta,q,\gamma_{m}\right)$ and $R\left(\rho\right)=2\max_{\tau\in\left\{0,a,b\right\}}\mathbb{E}\left\{\left|X_{\left(\tau\right)}\right|\right\}+2\rho+2C_{a,b}$. Since

[TABLE]

a union bound for probability implies that

[TABLE]

with probability at least $1-p_{m}^{\ast}\left(\vartheta,q,\gamma_{m}\right)$ if

[TABLE]

Part III: to determine the constants $\gamma$, $q$, $\vartheta$ and $\vartheta^{\prime}$ and a uniform consistency class. Recall $\gamma_{m}=\gamma\ln m$. Set $\gamma$, $\vartheta$ and $q$ such that $q\gamma>\vartheta>2^{-1}$ and $0\leq\vartheta^{\prime}<\vartheta-1/2$. Then $p_{m}^{\ast}\left(\vartheta,q,\gamma_{m}\right)\rightarrow 0$ and $m^{\vartheta-1/2}\gamma_{m}^{-1/2}\ln\gamma_{m}\rightarrow\infty$ as $m\rightarrow\infty$. If additionally $R\left(\rho\right)=O\left(m^{\vartheta^{\prime}}\right)$, then (B.11) holds. Recall $u_{m}=\min_{\tau\in\left\{a,b\right\}}\min_{\left\{j:\mu_{j}\neq\tau\right\}}\left|\mu_{j}-\tau\right|$. From inequality (B.7) in the proof of Theorem 4.2, we see that

[TABLE]

when $\tau_{m}^{-1}\left(1+u_{m}^{-1}\right)=o\left(\pi_{1,m}\right)$. So, when in addition $\tau_{m}\pi_{1,m}^{-1}\Upsilon\left(0,q,\tau_{m},\gamma_{m}\right)\rightarrow 0$, we see from (B.10) that

[TABLE]

In other words, as claimed,

[TABLE]

is a uniform consistency class.

Appendix C Proofs Related to Construction II for Type I location-shift family

C.1 Proof of Theorem 4.4

Since $F_{0}\left(x\right)$ is differentiable in $x$ and $\int\left|x\right|dF_{\mu}\left(x\right)<\infty$ for all $\mu\in U$, $r_{0}\left(t\right)$ is differentiable in $t\in\mathbb{R}$. Assume (4.10), i.e.,

[TABLE]

Then

[TABLE]

and

[TABLE]

is well-defined and equal to the left-hand side (LHS) of (C.1). Further,

[TABLE]

where we have invoked Fubini’s theorem due to (4.10) and the identity

[TABLE]

to obtain the second and third equalities, and the condition $\int\left|x\right|dF_{\mu}\left(x\right)<\infty$ for all $\mu\in U$ and $\left|s\right|\leq 1$ to assert

[TABLE]

to obtain the fourth equality. In other words, we have shown

[TABLE]

However, since $\mathcal{F}$ is a Type I location-shift family, we must have $\hat{F}_{\mu}\left(t\right)=\hat{F}_{0}\left(t\right)\exp\left(\iota t\mu\right)$ and $\hat{F}_{0}=r_{0}$. Therefore, the RHS (i.e., right-hand side) of (C.2) is equal to

[TABLE]

Namely,

[TABLE]

Since $\psi_{1}$ is real, $\psi_{1}\left(t,\mu\right)=\int K_{1}\left(t,x\right)dF_{\mu}\left(x\right)$ has to hold, where $K_{1}\left(t,x\right)=\Re\left\{K_{1}^{\dagger}\left(t,x\right)\right\}$. By Theorem 2.1, the pair (4.12) is as desired.

It is easy to verify that $\partial_{t}\left\{1/r_{0}\left(t\right)\right\}$ being odd (or even) in $t$ implies that both $\partial_{s}\left\{1/r_{0}\left(ys\right)\right\}$ and $\partial_{s}\left\{1/r_{0}\left(tys\right)\right\}$ are odd (or even) in $s$. Let $\tilde{r}_{0}^{\prime}\left(ys\right)=\partial_{s}\left\{1/r_{0}\left(ys\right)\right\}$. Then

[TABLE]

So, when $\tilde{r}_{0}^{\prime}\left(ys\right)$ is odd in $s$,

[TABLE]

and when $\tilde{r}_{0}^{\prime}\left(ys\right)$ is even in $s$,

[TABLE]

C.2 Proof of Theorem 4.5

First, we derive an upper bound for $\mathbb{V}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}$, where $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$ and $\varphi_{1,m}\left(t,\boldsymbol{\mu}\right)=\mathbb{E}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}$. Assume that $\partial_{t}\left\{1/r_{0}\left(t\right)\right\}$ is odd in $t$ and set $\tilde{r}_{0}\left(tys\right)=y^{-1}\partial_{s}\left\{1/r_{0}\left(tys\right)\right\}$. Then

[TABLE]

Define

[TABLE]

Then $\mathbb{V}\left\{\tilde{S}_{1,m,0}\left(ts,y\right)\right\}\leq 4m^{-1}$ and

[TABLE]

where $\sigma_{i}^{2}$ is the variance of $z_{i}$ and $\mu_{i}$ the mean of $z_{i}$.

Further, $\mathbb{V}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}\leq 2\tilde{I}_{1,m,0}+2\tilde{I}_{1,m,1}$, where

[TABLE]

and

[TABLE]

Set

[TABLE]

Then,

[TABLE]

and

[TABLE]

Therefore,

[TABLE]

When $\sigma_{i}^{2}=\sigma^{2}$ for all $i\in\left\{1,\ldots,m\right\}$, $\tilde{D}_{m}$ becomes $D_{m}=\sigma^{2}+m^{-1}\sum_{i=1}^{m}\mu_{i}^{2}$.

Secondly, we derive an upper bound for $\mathbb{V}\left\{\hat{\varphi}_{m}\left(t,\mathbf{z}\right)\right\}$. Recall $K\left(t,x\right)=2^{-1}-K_{1}\left(t,x\right)-2^{-1}K_{1,0}\left(t,x;0\right)$ and for $\tau\in U$

[TABLE]

By Theorem 2 of Chen (2019),

[TABLE]

where $g\left(t,\mu\right)=\int_{\left[-1,1\right]}r_{\mu}^{-1}\left(ts\right)ds$. So, combining (C.5) with (C.6) gives

[TABLE]

If $X\sim\mathcal{N}_{1}\left(\mu,\sigma^{2}\right)$, then $r_{\mu}^{-1}\left(tys\right)=\exp\left(2^{-1}t^{2}y^{2}s^{2}\sigma^{2}\right)$ and

[TABLE]

So, when $z_{i}\sim\mathcal{N}_{1}\left(\mu_{i},\sigma^{2}\right)$, we see from (C.4) that $\bar{r}_{0}\left(t\right)\leq t^{2}\sigma^{2}\exp\left(2^{-1}t^{2}\sigma^{2}\right)$ and $\check{r}_{0}\left(t\right)\leq\exp\left(2^{-1}t^{2}\sigma^{2}\right)$. Therefore, from (C.7) we obtain

[TABLE]

Thirdly, we show the second claim. Let $\lambda>0$ be a fixed constant and take $t>0$. Then

[TABLE]

by Hoeffding’s inequality, and (C.3) implies

[TABLE]

On the other hand, Theorem 2 of Chen (2019) states

[TABLE]

So, the definition

[TABLE]

implies that, when $\sigma_{i}^{2}=\sigma^{2}$ for all $i\in\left\{1,\ldots,m\right\}$,

[TABLE]

with probability at least

[TABLE]

If $z_{i}\sim\mathcal{N}_{1}\left(\mu_{i},\sigma^{2}\right)$, then

[TABLE]

and

[TABLE]

and

[TABLE]

So, (C.8) becomes

[TABLE]

with probability at least $q_{m}^{\ast}\left(\lambda\right)$.

Finally, we prove the consistency. Replace $\lambda$ and $t$ respectively in (C.10) and (C.9) by some positive sequences $\lambda_{m}\rightarrow 0$ and $t_{m}\rightarrow\infty$ to be determined later. We see from (C.10) that

[TABLE]

with probability at least $q_{m}^{\ast}\left(\lambda_{m}\right)$. Set $t_{m}=\sqrt{2\gamma\sigma^{-2}\ln m}$ for some $\gamma\in\left(0,0.5\right)$ and $\lambda_{m}=m^{-\gamma^{\prime}}$ for some $\gamma<\gamma^{\prime}<0.5$. Then, when $m$ is large enough,

[TABLE]

On the other hand, $t_{m}^{-1}\left(1+\tilde{u}_{m}^{-1}\right)=o\left(\pi_{1,m}\right)$ with $\tilde{u}_{m}=\min_{\left\{j:\mu_{j}\neq 0\right\}}\left|\mu_{j}\right|$ implies the following: $\pi_{1,m}^{-1}\varphi_{m}\left(t_{m},\boldsymbol{\mu}\right)\rightarrow 1$ (by similar reasoning that leads to inequality (B.7)), $t_{m}\pi_{1,m}\rightarrow\infty$ and $m^{\gamma-\gamma^{\prime}}=o\left(\pi_{1,m}\right)$, which implies

[TABLE]

and forces (C.11) to induce $\pi_{1,m}^{-1}\left|\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)-\varphi_{m}\left(t_{m},\boldsymbol{\mu}\right)\right|=o\left(1\right)$. If in addition $m^{-1}\sum_{i=1}^{m}\mu_{i}^{2}=o\left(m^{1-2\gamma^{\prime}}\right)$, then $\check{q}_{m}\left(\lambda_{m}\right)$ in (C.12) is $o\left(1\right)$. Thus, $\pi_{1,m}^{-1}\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)\rightarrow 1$ with probability $1-\check{q}_{m}\left(\lambda_{m}\right)=1-o\left(1\right)$. In other words, a uniform consistency class is

[TABLE]

Appendix D Proofs Related to Construction I for the Gamma family

D.1 Proof of Theorem 5.1

Recall $\tilde{c}_{n}\left(\theta\right)=\int x^{n}dG_{\theta}\left(x\right)$ and define

[TABLE]

By assumption, $\tilde{c}_{n}\left(\theta\right)=\xi^{n}\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{n}=\zeta_{0}\xi^{n}\left(\theta\right)\tilde{a}_{n}$, where $\zeta_{0}\equiv\zeta\equiv 1$. So, $\mu\left(\theta\right)=\xi\left(\theta\right)\zeta\left(\theta\right)\tilde{a}_{1}=\xi\left(\theta\right)\zeta_{0}\tilde{a}_{1}$ and

[TABLE]

Since $\psi_{1}$ is real, $\psi_{1}=\mathbb{E}\left\{\Re\left(K_{1}^{{\dagger}}\right)\right\}$. However,

[TABLE]

Since $\mu\left(\theta\right)$ is smooth and strictly increasing in $\theta\in\Theta$, $a\leq\mu\leq b$ if and only if $\theta_{a}\leq\theta\leq\theta_{b}$. By Theorem 2.1, the pair $\left(K,\psi\right)$ in (5.3) is as desired.

D.2 Proof of Theorem 5.2

In order the present the proof, we quote Lemma 4 of Chen (2019) as follows: for a fixed $\sigma>0$, let

[TABLE]

If $Z$ has CDF $G_{\theta}$ from the Gamma family with scale parameter $\sigma$, then

[TABLE]

for positive and sufficiently large $z$.

Now we present the arguments. Take $t>0$ to be sufficiently large. Firstly, we will obtain an upper bound for $\mathbb{V}\left\{\hat{\varphi}_{m}\left(t,\mathbf{z}\right)\right\}$. For Gamma family, $\zeta\left(\theta\right)\equiv\zeta_{0}=1$, $\tilde{a}_{1}=\sigma$ and $\mu\left(\theta\right)=\sigma\xi\left(\theta\right)$. Define

[TABLE]

and set $S_{1,m}\left(t\right)=m^{-1}\sum_{i=1}^{m}\left[w_{1}\left(t,z_{i}\right)-\mathbb{E}\left\{w_{1}\left(t,z_{i}\right)\right\}\right]$. Then

[TABLE]

Define $\tilde{V}_{1,m}=\mathbb{V}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}$, where $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$ and $\varphi_{1,m}\left(t,\boldsymbol{\theta}\right)=\mathbb{E}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}$. Then,

[TABLE]

Since $\left|w_{1}\left(t,x\right)\right|\leq\Gamma\left(\sigma\right)\tilde{w}\left(t\sigma,x\right)$ uniformly in $\left(t,x\right)$, the inequality (D.3) implies

[TABLE]

where we recall $u_{3,m}=\min_{1\leq i\leq m}\left\{1-\theta_{i}\right\}$ and have set

[TABLE]

Recall for $\tau\in\left\{a,b\right\}$

[TABLE]

Define $\hat{\varphi}_{3,0,m}\left(t,\mathbf{z};\tau\right)=m^{-1}\sum_{i=1}^{m}K_{3,0}\left(t,z_{i};\theta_{\tau}\right)$ and $\varphi_{3,0,m}\left(t,\boldsymbol{\theta};\tau\right)=\mathbb{E}\left\{\hat{\varphi}_{3,0,m}\left(t,\mathbf{z};\tau\right)\right\}$. Then Theorem 8 of Chen (2019) implies

[TABLE]

So,

[TABLE]

where

[TABLE]

Secondly, we provide a uniform consistency class. If $\sigma\geq 11/4$, then (D.5) induces

[TABLE]

where $\left\|1-\boldsymbol{\theta}\right\|_{\infty}=\max_{1\leq i\leq m}\left(1-\theta_{i}\right)$. Let $\varepsilon>0$ be a constant and set $t=\left(4\sigma\right)^{-1}u_{3,m}\gamma\ln m$ for any fixed $\gamma\in\left(0,1\right]$. Then, (D.4) and (D.6) imply

[TABLE]

In contrast, if $\sigma\leq 3/4$, then (D.5) implies

[TABLE]

Set $t=4^{-1}u_{3,m}\gamma\ln m$ for any fixed $\gamma\in\left(0,1\right)$. Then

[TABLE]

To determine a uniform consistency class, we only need to incorporate the speed of convergence of the Oracle. Recall for $\tau\in\left\{a,b\right\}$

[TABLE]

By the same reasoning that leads to (B.7), we have

[TABLE]

where $\tilde{u}_{3,m}=\min_{\tau\in\left\{a,b\right\}}\min_{\left\{j:\theta_{j}\neq\theta_{\tau}\right\}}\left|\xi\left(\theta_{\tau}\right)-\xi\left(\theta_{i}\right)\right|$. So, $\pi_{1,m}^{-1}\varphi_{m}\left(t,\boldsymbol{\mu}\right)\rightarrow 1$ if $t^{-1}\left(1+\tilde{u}_{3,m}^{-1}\right)=o\left(\pi_{1,m}\right)$. Therefore, by (D.7) a uniform consistency class when $\sigma>11/4$ is

[TABLE]

for each $\gamma\in(0,1]$, and by (D.8) a uniform consistency class when $\sigma\leq 3/4$ is

[TABLE]

for each $\gamma\in\left(0,1\right)$.

Appendix E Proofs Related to Construction II for the Gamma family

E.1 Proof of Theorem 5.3

Recall $\tilde{c}_{n}\left(\theta\right)=\int x^{n}dG_{\theta}\left(x\right)=\zeta_{0}\xi^{n}\left(\theta\right)\tilde{a}_{n}$ for the constant $\zeta_{0}=1$ and $\mu\left(\theta\right)=\tilde{c}_{1}$. Set

[TABLE]

Then

[TABLE]

Further,

[TABLE]

On the other hand, set

[TABLE]

Then

[TABLE]

Further,

[TABLE]

Set $K_{1}^{\dagger}\left(t,x\right)=K_{4,0}^{\dagger}\left(t,x\right)+K_{4,1}^{\dagger}\left(t,x\right)$. Then

[TABLE]

and

[TABLE]

Since $\psi_{1}\left(t,\theta\right)$ is real-valued, we also have $\psi_{1}\left(t,\theta\right)=\int K_{1}\left(t,x\right)dG_{\theta}\left(x\right)$, where

[TABLE]

Now set $K\left(t,x\right)=2^{-1}-K_{1}\left(t,x\right)-2^{-1}K_{0}\left(t,x;\theta_{b}\right)$ with

[TABLE]

given by Theorem 2.1. Then

[TABLE]

By Theorem 2.1 the pair $\left(K,\psi\right)$ in (5.8) is as desired.

E.2 Proof of Theorem 5.4

We need the following:

Lemma E.1**.**

For a fixed $\sigma>0$, let

[TABLE]

If $Z$ has CDF $G_{\theta}$ from the Gamma family with scale parameter $\sigma$, then

[TABLE]

for positive and sufficiently large $z$.

The proof of Lemma E.1 is provided in Section E.3. Now we provide the arguments. Take $t>0$ to be sufficiently large. First, we obtain an upper bound on $\mathbb{V}\left\{\hat{\varphi}_{m}\left(t,\mathbf{z}\right)\right\}$. Note that $\zeta_{0}=1$ and $\tilde{a}_{1}=\sigma$. For $y\in\left[0,1\right]$ and $t,x>0$, define

[TABLE]

and

[TABLE]

Then, uniformly for $s\in\left[-1,1\right]$ and $y\in\left[0,1\right]$,

[TABLE]

and

[TABLE]

Set $\hat{S}_{3,m,1}\left(ts,y\right)=m^{-1}\sum_{i=1}^{m}w_{3,1}\left(ts,x,y\right)$, $\hat{S}_{3,m,2}\left(ts,y\right)=bm^{-1}\sum_{i=1}^{m}w_{3,2}\left(ts,x,y\right)$ and

[TABLE]

Recall

[TABLE]

$\vskip 12.0pt plus 4.0pt minus 4.0pt\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$ and $\varphi_{1,m}\left(t,\boldsymbol{\theta}\right)=m^{-1}\sum_{i=1}^{m}\mathbb{E}\left\{K_{1}\left(t,z_{i}\right)\right\}$. Then,

[TABLE]

By the inequalities (E.2), (E.3), (D.3) and Lemma E.1, we have

[TABLE]

and

[TABLE]

where $u_{3,m}=\min_{1\leq i\leq m}\left\{1-\theta_{i}\right\}$. Combining (E.4), (E.5) and (E.6) gives

[TABLE]

where

[TABLE]

Recall

[TABLE]

and $\hat{\varphi}_{3,0,m}\left(t,\mathbf{z};\theta_{b}\right)=m^{-1}\sum_{i=1}^{m}K_{3,0}\left(t,z_{i};\theta_{b}\right)$ and $\varphi_{3,0,m}\left(t,\boldsymbol{\theta};\tau\right)=\mathbb{E}\left\{\hat{\varphi}_{3,0,m}\left(t,\mathbf{z};\theta_{b}\right)\right\}$. Then Theorem 8 of Chen (2019) asserts

[TABLE]

Recall $K\left(t,x\right)=2^{-1}-K_{1}\left(t,x\right)-2^{-1}K_{3,0}\left(t,x;\theta_{b}\right)$. Then

[TABLE]

Secondly, we provide a uniform consistency class. If $\sigma\geq 11/4$, then (E.8) and (E.7) imply

[TABLE]

whereas if $\sigma\leq 2^{-1}\sqrt{2}$, then (E.8) and (E.7) imply

[TABLE]

Let $\varepsilon>0$ be a constant. Set $t_{m}=\left(4\sqrt{2}\sigma\right)^{-1}u_{3,m}\gamma\ln m$ for any fixed $\gamma\in(0,1]$ for (E.9) gives

[TABLE]

when $\sigma\geq 11/4$, whereas setting $t_{m}=4^{-1}u_{3,m}\gamma\ln m$ for any fixed $\gamma\in\left(0,1\right)$ for (E.10) gives also (E.11) but with $\tilde{l}\left(\boldsymbol{\theta},\sigma\right)$ given by (E.10) when $\sigma\leq 2^{-1}\sqrt{2}$. Recall

[TABLE]

By the same reasoning that leads to (B.7), we have

[TABLE]

where $\check{u}_{3,m}=\min_{\left\{j:\theta_{j}\neq\theta_{b}\right\}}\left|\xi\left(\theta_{b}\right)-\xi\left(\theta_{i}\right)\right|$. So, $\pi_{1,m}^{-1}\varphi_{m}\left(t_{m},\boldsymbol{\mu}\right)\rightarrow 1$ when $t_{m}^{-1}\left(1+\check{u}_{3,m}^{-1}\right)=o\left(\pi_{1,m}\right)$. Therefore, a uniform consistency class is

[TABLE]

for $\sigma\geq 11/4$, $\gamma\in(0,1]$ and $\tilde{l}\left(\boldsymbol{\theta},\sigma\right)$ given by (E.9), and it is

[TABLE]

for $\sigma\leq 2^{-1}\sqrt{2}$, $\gamma\in\left(0,1\right)$ and $\tilde{l}\left(\boldsymbol{\theta},\sigma\right)$ given by (E.10).

E.3 Proof of Lemma E.1

Recall (D.2), i.e.,

[TABLE]

From the proof of Lemma 4 of Chen (2019), we have

[TABLE]

when $zx\rightarrow\infty$. So, when $zx\rightarrow\infty$,

[TABLE]

Let $A_{1,z}=\left\{x\in\left(0,\infty\right):zx=O\left(1\right)\right\}$. Then, on the set $A_{1,z}$, $f_{\theta}\left(x\right)=O\left(x^{\sigma-1}\right)$ and $\tilde{w}\left(z,x\right)\leq Ce^{zx}=O\left(1\right)$ when $\theta<1$. Therefore,

[TABLE]

On the other hand, let $A_{2,z}=\left\{x\in\left(0,\infty\right):\lim_{z\rightarrow\infty}zx=\infty\right\}$. Then

[TABLE]

where

[TABLE]

By the formula,

[TABLE]

we have

[TABLE]

By Stirling’s formula,

[TABLE]

Therefore,

[TABLE]

where $Q^{\ast}\left(z\right)=\sum_{n=0}^{\infty}\frac{z^{n/2}}{\sqrt{n!}}$. By definition (8.01) and identity (8.07) in Chapter 8 of Olver (1974),

[TABLE]

Combining (E.13) through (E.15) gives

[TABLE]

for all positive and sufficiently large $z$. Recall (E.12). Thus, when $1-\theta>0$, $\sigma>0$ and $z$ is positive and sufficiently large,

[TABLE]

Appendix F Proofs Related to the Extension

F.1 Proof of Lemma 6.1

Firstly,

[TABLE]

Namely, setting $\hat{\phi}\left(s\right)=\int_{a}^{b}\phi\left(y\right)\exp\left(-\iota ys\right)dy$ yields

[TABLE]

To show the second claim, we first prove the following: if $\phi$ is of bounded variation on $\left[0,\delta\right]$ with $\delta>0$ and $t>0$, then

[TABLE]

Without loss of generality, we can assume $\phi\left(0\right)=0$ and that $\phi$ is non-negative and non-decreasing. Clearly, $\lim_{y\rightarrow 0}y^{-1}\sin\left(ty\right)=t$ implies $y^{-1}\sin\left(ty\right)\in L^{1}\left(\left[0,\delta\right]\right)$ for each $t\neq 0$. By the second law of the mean,

[TABLE]

for some $\delta^{\prime}\in\left[0,\delta\right]$. However, Lemma A.2 implies

[TABLE]

For the general setting where $\phi$ is of bounded variation, the Jordan decomposition $\phi=\phi_{1}-\phi_{2}$ holds such that both $\phi_{1}$ and $\phi_{2}$ are non-decreasing functions on $\left[0,\delta\right]$. We obtain (F.1) by applying (F.2) to $\tilde{\phi}_{1}$ and $\tilde{\phi}_{2}$, each defined as $\tilde{\phi}_{i}\left(\cdot\right)=\phi_{i}\left(\cdot\right)-\phi_{i}\left(0\right)$ on $\left[0,\delta\right]$ for $i\in\left\{1,2\right\}$.

Now we show the second claim. Take $t>0$. Since $\left\|\phi\right\|_{\mathrm{TV}}<\infty$ and $0<b-a<\infty$, then $\phi\in L^{1}\left(\left[a,b\right]\right)$. Obviously,

[TABLE]

We split the rest of the arguments into 2 cases: (Case 1) if $\mu=a$ or $b$, then (F.1) implies, when $t\geq 2$,

[TABLE]

On the other hand, if $a<\mu<b$, then

[TABLE]

and (F.1) implies

[TABLE]

(Case 2) Since $\phi$ is of bounded variation, we can assume that $\phi$ is non-decreasing as did previously. If $\mu<a$ or $\mu>b$, then

[TABLE]

for some $\delta^{\prime}\in\left[a,b\right]$. Applying Lemma A.2 to the RHS of (F.5), we have

[TABLE]

Combining (F.3), (F.4) and (F.6) gives

[TABLE]

F.2 Proof of Theorem 6.1

We show the first claim. Recall $\hat{\phi}\left(s\right)=\int_{a}^{b}\phi\left(y\right)\exp\left(-\iota ys\right)dy$ and

[TABLE]

Set

[TABLE]

Then

[TABLE]

Namely, $\int K_{1}\left(t,x\right)dF_{\mu}\left(x\right)=\mathcal{D}_{g}\left(t,\mu;a,b\right)$ as desired. Since $\mathcal{F}$ is a Type I location-shift family, $\hat{F}_{0}\equiv r_{0}$ holds, $r_{0}$ is an even function, and

[TABLE]

Finally, we only need to capture the contributions of the end points $a$ and $b$ to estimating $\check{\pi}_{0,m}$. By Theorem 2.1, we only need to set $\left(K,\psi\right)$ as given by (6.7).

Now we show the second claim. Recall $\tilde{c}_{n}\left(\theta\right)=\int x^{n}dG_{\theta}\left(x\right)=\zeta_{0}\xi^{n}\left(\theta\right)\tilde{a}_{n}$ and $\mu\left(\theta\right)=\zeta_{0}\xi\left(\theta\right)\tilde{a}_{1}$ and $\zeta_{0}=1$. Define

[TABLE]

Then,

[TABLE]

Since $\psi_{1}$ is real, $\psi_{1}=\mathbb{E}\left\{\Re\left(K_{1}^{{\dagger}}\right)\right\}$. However,

[TABLE]

By Theorem 2.1, the pair $\left(K,\psi\right)$ in (6.10) is as desired.

F.3 Proof of Theorem 6.2

The proof uses almost identical arguments as those for the proofs of Theorem 4.2 and Theorem 5.2. So, we only provide the key steps. Take $t>0$. The rest of the proof is divided into 2 parts: one for Type I location-shift family and the other for Gamma family.

Part I “the case of a Type I location-shift family”: Recall

[TABLE]

and $\psi_{1}\left(t,\mu\right)=\int K_{1}\left(t,x\right)dF_{\mu}\left(x\right)=\mathcal{D}_{\phi}\left(t,\mu;a,b\right)$. Take $t\geq 2$. Following the proof of Theorem 4.2, we immediately see that

[TABLE]

and

[TABLE]

where $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$. Combining (F.7) and (B.5) gives

[TABLE]

and combining (F.8) and (B.4) gives

[TABLE]

with probability at least $1-4\exp\left(-2^{-1}\lambda^{2}\right)$. Recall

[TABLE]

from Theorem 2.1 and $u_{m}=\min_{\tau\in\left\{a,b\right\}}\min_{\left\{j:\mu_{j}\neq\tau\right\}}\left|\mu_{j}-\tau\right|$. Then, Lemma A.2, Lemma A.4 and Lemma 6.1 together imply

[TABLE]

and $\Pr\left\{\check{\pi}_{0,m}^{-1}\hat{\varphi}_{m}\left(t_{m},\mathbf{z}\right)\rightarrow 1\right\}\rightarrow 1$ when $t_{m}^{-1}\left(1+u_{m}^{-1}\right)=o\left(\check{\pi}_{0,m}\right)$, $\lambda_{m}m^{-1/2}g\left(t,0\right)t_{m}=o\left(\check{\pi}_{0,m}\right)$ and $\exp\left(-2^{-1}\lambda_{m}^{2}\right)\rightarrow 0$.

Part II “the case of Gamma family”: Recall

[TABLE]

and

[TABLE]

Take $t>0$ to be sufficiently large. Following the proof of Theorem 5.2, we have

[TABLE]

Recall $u_{3,m}=\min_{1\leq i\leq m}\left\{1-\theta_{i}\right\}$. Then, when $\sigma\geq 11/4$, (F.9) implies

[TABLE]

by setting $t=4^{-1}\sigma^{-1}u_{3,m}\gamma\ln m$ for any fixed $\gamma\in\left(0,1\right]$, whereas, when $\sigma\leq 3/4$, (F.9) implies

[TABLE]

by setting $t=4^{-1}u_{3,m}\gamma\ln m$ for any fixed $\gamma\in\left(0,1\right)$.

Finally, recall $\tilde{u}_{3,m}=\min_{\tau\in\left\{a,b\right\}}\min_{\left\{j:\theta_{j}\neq\theta_{\tau}\right\}}\left|\xi\left(\theta_{\tau}\right)-\xi\left(\theta_{i}\right)\right|$ and for $\tau\in\left\{a,b\right\}$

[TABLE]

Then, by the lemmas on Dirichlet integral,

[TABLE]

and $\check{\pi}_{0,m}^{-1}\varphi_{m}\left(t,\boldsymbol{\mu}\right)\rightarrow 1$ when $t^{-1}\left(1+\tilde{u}_{3,m}^{-1}\right)=o\left(\check{\pi}_{0,m}\right)$. Therefore, from (F.10) and (F.11), we obtain the claimed uniform consistency class for $\sigma\geq 11/4$ and $\sigma\leq 3/4$ respectively.

F.4 Proof of Corollary 6.1

Take $t\geq 2$. Recall $e_{m}\left(t\right)=\hat{\varphi}_{m}\left(t,\mathbf{z}\right)-\varphi_{m}\left(t,\boldsymbol{\mu}\right)$, $g\left(t,\mu\right)=\int_{\left[-1,1\right]}\left\{1/r_{\mu}\left(ts\right)\right\}ds$, and $\gamma_{m}=\gamma\ln m$. Following almost identical arguments in Part I and II of the proof of Theorem 4.3, we can obtain

[TABLE]

with probability at least $1-p_{m}^{\ast}\left(\vartheta,q,\gamma_{m}\right)$ if $\tau_{m}\leq\gamma_{m}$ and

[TABLE]

where $p_{m}^{\ast}\left(\vartheta,q,\gamma_{m}\right)$ and $R\left(\rho\right)$ are defined in the proof of Theorem 4.3. The fact that

[TABLE]

implies

[TABLE]

when $\tau_{m}=\sqrt{2\gamma\sigma^{-2}\ln m}$ is set for $\gamma\in\left(0,0.5\right)$. The RHS of (F.13) is $o\left(1\right)$ when $m^{\gamma-0.5}=o\left(\check{\pi}_{0,m}\right)$. By almost identical arguments in Part III of the proof of Theorem 4.3, setting $\gamma$, $\vartheta$, $\vartheta^{\prime}$ and $q$ such that $q\gamma>\vartheta>2^{-1}$ and $0\leq\vartheta^{\prime}<\vartheta-1/2$ forces $p_{m}^{\ast}\left(\vartheta,q,\gamma_{m}\right)\rightarrow 0$ and $m^{\vartheta-1/2}\gamma_{m}^{-1/2}\ln\gamma_{m}\rightarrow\infty$ as $m\rightarrow\infty$. If additionally $R\left(\rho\right)=O\left(m^{\vartheta^{\prime}}\right)$, then (F.12) holds.

On the other hand, we recall $u_{m}=\min_{\tau\in\left\{a,b\right\}}\min_{\left\{j:\mu_{j}\neq\tau\right\}}\left|\mu_{j}-\tau\right|$ and have

[TABLE]

So, $\check{\pi}_{0,m}^{-1}\varphi_{m}\left(t_{m},\mathbf{z}\right)\rightarrow 1$ when $t_{m}^{-1}\left(1+u_{m}^{-1}\right)=o\left(\check{\pi}_{0,m}\right)$. Set $t_{m}=\tau_{m}$. Then $m^{\gamma-0.5}=o\left(\check{\pi}_{0,m}\right)$ when $t_{m}^{-1}=o\left(\check{\pi}_{0,m}\right)$. So, a uniform consistency class is

[TABLE]

The second assertion of the corollary holds easily as argued as follows. Recall the estimator $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)=m^{-1}\sum_{i=1}^{m}K_{1}\left(t,z_{i}\right)$ with $K_{1}$ in (6.5). Set $\varphi_{1,m}\left(t,\mathbf{z}\right)=\mathbb{E}\left\{\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)\right\}$. Then the lemmas on Dirichlet integral imply

[TABLE]

On the other hand, it is easy to see that (F.13) remains valid for $\hat{\varphi}_{1,m}\left(t,\mathbf{z}\right)$. So, a uniform consistency class is

[TABLE]

Appendix G Simulation study

We will present a simulation study on the proposed estimators, with a comparison to the “MR” estimator of Meinshausen and Rice (2006) or Storey’s estimator of Storey et al. (2004) for the case of a one-sided. For one-sided null $\Theta_{0}=\left(-\infty,b\right)\cap U$, when $X_{0}$ is an observation from a random variable $X$ with CDF $F_{\mu}$, $\mu\in U$, its one-sided p-value is computed as $1-F_{b}\left(X_{0}\right)$. For one-side null on the means of Gaussian random variables, we will also compare our estimator with one minus the null proportion estimator of Section 4.3 of Hoang and Dickhaus (2021a) that has its first tuning parameter (for thresholding randomized p-values) as $0.5$ and second tuning parameter $\tilde{c}_{0}$ (for constructing randomized p-values), which leads to the “HD” estimator with $\tilde{c}_{0}=0.5$ and the “HD1” estimator with the value for $\tilde{c}_{0}$ that is optimally determined from data from a grid of $50$ equally spaced candidate values for $\tilde{c}_{0}$. For other settings of our simulation study (that are given below and in Section G.2), we will not include a comparison with the “HD” or “HD1” estimator, since it is not an aim here to investigate for these other settings whether the definition of randomized p-value of Dickhaus (2013); Hoang and Dickhaus (2021b, a) leads to valid randomized p-values that can be practically computed.

We numerically implement the solution $\left(\psi,K\right)$ in two cases as follows: (a) if $\psi$ or $K$ is defined by a univariate integral, then the univariate integral is approximated by a Riemann sum based on an equally spaced partition with norm $0.01$ of the corresponding domain of integration; (b) if $\psi$ or $K$ is defined by a double integral, then the double integral is computed as an iterated integral, for which each univariate integral is computed as if it were case (a). We choose norm $0.01$ for a partition so as to reduce a bit the computational complexity of the proposed estimators when the number of hypotheses to test is very large. However, we will not explore here how much more accurate these estimators can be when finer partitions are used to obtain the Riemman sums, or explore here which density function $\omega(s)$ on $[-1,1]$ should be used to give the best performances to the proposed estimators among all continuous densities on $[-1,1]$ that are of bounded variation. By default, we will choose the triangular density $\omega\left(s\right)=\left(1-|s|\right)1_{\left[-1,1\right]}\left(s\right)$, since numerical evidence in Jin (2008); Chen (2019) shows that this density leads to good performances of the proposed estimators for the setting of a point null.

The MR estimator (designed for continuous p-values) is implemented as follows: let the ascendingly ordered p-values be $p_{\left(1\right)}<p_{\left(2\right)}<\cdots<p_{\left(m\right)}$ for $m>4$, set $b_{m}^{\ast}=m^{-1/2}\sqrt{2\ln\ln m}$, and define

[TABLE]

then $\hat{\pi}_{1,m}^{\mathsf{MR}}=\min\left\{1,\max\left\{0,\max_{2\leq i\leq m-2}q_{i}^{\ast}\right\}\right\}$ is the MR estimator. Storey’s estimator will be implemented by the qvalue package (version 2.14.1) via the ‘pi0.method=smoother’ option. All simulations will be done with R version 3.5.0.

For an estimator $\hat{\pi}_{1,m}$ of $\pi_{1,m}$ or an estimator $\hat{\pi}_{0,m}$ of $\tilde{\pi}_{0,m}$, its accuracy is measured by the excess $\tilde{\delta}_{m}=\hat{\pi}_{1,m}\pi_{1,m}^{-1}-1$ or $\tilde{\delta}_{m}=\hat{\pi}_{0,m}\tilde{\pi}_{0,m}^{-1}-1$. For each experiment, the mean $\mu_{m}^{\ast}$ and standard deviation $\sigma_{m}^{\ast}$ of $\tilde{\delta}_{m}$ is estimated from independent realizations of the experiment. Among two estimators, the one that has smaller $\sigma_{m}^{\ast}$ is taken to be more stable, and the one that has both smaller $\sigma_{m}^{\ast}$ and smaller $\left|\mu_{m}^{\ast}\right|$ is better.

G.1 Simulation design and results for Gaussian random variables

We will simulate $\mathbf{z}\sim\mathcal{N}_{m}\left(\boldsymbol{\mu},\boldsymbol{\Sigma}\right)$ with $\boldsymbol{\Sigma}$ as the identity matrix. For $a<b$, let $\mathsf{U}\left(a,b\right)$ be the uniform random variable or the uniform distribution on the closed interval $\left[a,b\right]$. We consider $6$ values for $m=10^{3}$, $5\times 10^{3}$, $10^{4}$, $5\times 10^{4}$, $10^{5}$ or $5\times 10^{5}$, and $2$ sparsity levels $\pi_{1,m}=0.2$ (indicating the dense regime) or $\left(\ln\ln m\right)^{-1}$ (indicating the moderately sparse regime). The speed of the proposed estimators $t_{m}=\sqrt{0.99\ln m}$ (i.e., $t_{m}$ has tuning parameter $\gamma=0.495$) and $u_{m}=\tilde{u}_{m}=\left(\ln\ln m\right)^{-1}$, where $u_{m}$ and $\tilde{u}_{m}$ are respectively defined by (4.5) and Theorem 4.5. This ensures $t_{m}^{-1}\left(1+\max\left\{u_{m}^{-1},\tilde{u}_{m}^{-1}\right\}\right)=o\left(\pi_{1,m}\right)$ and the consistency of the proposed estimator as per Theorem 4.3 and Theorem 4.5. The simulated data are generated as follows:

• •

Scenario 1 “estimating $\pi_{1,m}$ for a bounded null”: set $a=-1$ and $b=2$; generate $m_{0}$ $\mu_{i}$’s independently from $\mathsf{U}\left(a+u_{m},b-u_{m}\right)$, $m_{11}$ $\mu_{i}$’s independently from $\mathsf{U}\left(b+u_{m},b+6\right)$, and $m_{11}$ $\mu_{i}$’s independently from $\mathsf{U}\left(a-4,a-u_{m}\right)$, where $m_{11}=\max\left\{1,\left[0.5m_{1}\right]-\left[m/\ln\ln m\right]\right\}$; set half of the remaining $m-m_{0}-2m_{11}$ $\mu_{i}$’s to be $a$, and the rest to be $b$.

• •

Scenario 2 “estimating $\pi_{1,m}$ for a one-sided null”: set $b=0$; generate $m_{0}$ $\mu_{i}$’s independently from $\mathsf{U}\left(-4,b-u_{m}\right)$, and $\left[0.9m_{1}\right]$ $\mu_{i}$’s independently from $\mathsf{U}\left(b+u_{m},b+6\right)$; set the rest $\mu_{i}$’s to be $b$.

• •

Scenario 3 “estimating average, truncated 2-norm”, i.e., estimating $\tilde{\pi}_{0,m}$ in (6.13) with $\phi\left(t\right)=\left|t\right|^{2}1_{\left\{\left|t\right|\leq b\right\}}\left(t\right)$ for a fixed $b>0$: set $b=2$; generate $m_{0}$ $\mu_{i}$’s independently from $\mathsf{U}\left(a,b\right)$, $\left[0.5m_{1}\right]$ $\mu_{i}$’s independently from $\mathsf{U}\left(b+u_{m},b+6\right)$, and the rest $\mu_{i}$’s independently from $\mathsf{U}\left(b-4,b-u_{m}\right)$. In this setting, $C^{-1}\pi_{1,m}\leq\tilde{\pi}_{1,m}\leq C\pi_{1,m}$ holds for some constant $C>0$ and $t_{m}^{-1}=o\left(\tilde{\pi}_{1,m}\right)$ holds, ensuring the consistency of the proposed estimator $\hat{\varphi}_{1,m}$ as per Corollary 6.1.

Scenario 1 models the setting that when testing a bounded null in practice, it is unlikely that there is always a positive proportion of means or medians that are equal to either of the two boundary points, and Scenario 2 takes into account that when testing a one-sided null with [math] as the boundary point, it is likely that there is a positive or diminishing proportion of means or medians that are equal to [math], as in differential gene expression studies. Each triple of $\left(m,\pi_{1,m},\Theta_{0}\right)$ or $\left(m,\tilde{\pi}_{0,m},\Theta_{0}\right)$ determines an experiment, and there are $36$ experiments in total. Each experiment is repeated independently $200$ times.

Figure G.1 visualizes the simulation results, for which Storey’s estimator is not shown since it is always [math] for all experiments in Scenario 2. Such a strange behavior of Storey’s estimator has not been reported before and is worth investigation but is not our focus here. A plausible explanation for this is that Storey’s estimator excessively over-estimates $\pi_{0,m}$ when no p-value is uniformly distributed under the null. The following five observations can be made: (i) for estimating the alternative proportion for a one-sided null, the proposed estimator is more accurate than the MR estimator, and it shows a strong trend of convergence towards consistency in the dense regime and a slow trend of convergence in the moderately sparse regime. (ii) for estimating the alternative proportion for a bounded null, the proposed estimator is accurate, and it shows a strong trend of convergence towards consistency in the dense regime but a slow convergence in the moderately sparse regime. (iii) the proposed estimator very accurately estimates the average, truncated $2$-norm, with a strong trend of convergence towards consistency. (iv) The MR estimator does not seem to actively capture the changes in the number of alternative hypotheses as the number of hypotheses varies. (v) For testing one-sided null on the means of Gaussian random variables, in terms of accuracy, the HD1 estimator performs worse than all of the HD, MR and proposed estimators and hence is not shown, whereas the HD estimator performs better than the MR estimator but worse than the proposed estimator. We remark that the accuracy and speed of convergence of the proposed estimators can be improved by employing more accurate Riemann sums for the integrals than currently used.

G.2 Simulation design and results for Gamma random variables

When implementing the estimator in Theorem 5.1 or Theorem 5.3, the power series in the definition of $K$ in (5.3) or (5.8) is replaced by the partial sum of its first $26$ terms, i.e., the power series is truncated at $n=25$. However, the double integral in $K$ in (5.3) or (5.8) has to be approximated by a Riemann sum (using the scheme described in the beginning of Appendix G) for each $z_{i}$ for a total of $m$ times. This greatly increases the computational complexity of applying $K$ to $\left\{z_{i}\right\}_{i=1}^{m}$ when $m$ is very large. So, we only consider $4$ values for $m$, i.e., $m=10^{3}$, $5\times 10^{3}$, $10^{4}$ or $5\times 10^{4}$, together with $2$ sparsity levels $\pi_{1,m}=0.2$ or $\left(\ln\ln m\right)^{-1}$. We set $\sigma=4$ for the simulated Gamma random variables. The speed of the proposed estimators $t_{m}=\sqrt{0.25\sigma^{-1}u_{3,m}\ln m}$ (i.e., $\gamma=1$ is set for $t_{m}$) for a bounded null and $t_{m}=2^{-5/4}\sigma^{-1/2}\sqrt{u_{3,m}\ln m}$ (i.e., $\gamma=1$ is set for $t_{m}$) for a one-side null, both with $u_{3,m}=0.2/\ln\ln m$, so that the consistency conditions in Theorem 5.2 and Theorem 5.4 are satisfied. The simulated data are generated as follows:

• •

Scenario 4 “estimating $\pi_{1,m}$ for a bounded null”: set $\theta_{a}=0$, $\theta_{b}=0.35$, $\theta_{\ast}=-0.2$ and $\theta^{\ast}=0.55$; generate $m_{0}$ $\theta_{i}$’s independently from $\mathsf{U}\left(\theta_{a}+u_{3,m},\theta_{b}-u_{3,m}\right)$, $m_{11}$ $\theta_{i}$’s independently from $\mathsf{U}\left(\theta_{b}+u_{m},\theta^{\ast}\right)$, and $m_{11}$ $\theta_{i}$’s independently from $\mathsf{U}\left(\theta_{\ast},\theta_{a}-u_{m}\right)$, where $m_{11}=\max\left\{1,\left[0.5m_{1}\right]-\left[m/\ln\ln m\right]\right\}$; set half of the remaining $m-m_{0}-2m_{11}$ $\theta_{i}$’s to be $\theta_{a}$, and the rest to be $\theta_{b}$.

• •

Scenario 5 “estimating $\pi_{1,m}$ for a one-sided null”: generate $m_{0}$ $\mu_{i}$’s independently from $\mathsf{U}\left(\theta_{\ast},\theta_{b}-u_{m}\right)$, and $\left[0.9m_{1}\right]$ $\mu_{i}$’s independently from $\mathsf{U}\left(\theta_{b}+u_{m},\theta^{\ast}\right)$; set the rest $\theta_{i}$’s to be $\theta_{b}$.

Each triple of $\left(m,\pi_{1,m},\Theta_{0}\right)$ determines an experiment, and there are $20$ experiments in total. Each experiment is repeated independently $100$ times. The assessment method for an estimator $\hat{\pi}_{1,m}$ of $\pi_{1,m}$ is again based on the mean and standard deviation of the excess $\tilde{\delta}_{m}=\hat{\pi}_{1,m}\pi_{1,m}^{-1}-1$, as was done for the simulations involving Gaussian random variables. Figure G.2 visualizes the simulation results, for which Storey’s estimator is not shown since it is always [math] for all experiments in Scenario 5 (similar to the setting of Gaussian random variables).

The following three observations can be made: (i) for estimating the alternative proportion for a one-sided null, the proposed estimator is much more accurate than the MR estimator, is very stable, and shows a clear trend of convergence towards consistency. In contrast, the MR estimator is always very close to [math], either failing to detect the existence of alternative hypotheses or very inaccurately estimating the alternative proportion. (ii) for estimating the alternative proportion for a bounded null, the proposed estimator is stable and reasonably accurate, and shows a clear trend of convergence towards consistency. (iii) the proposed estimator seems to be much more accurate in the moderately sparse regime than in the dense regime. Similar to the case of Gaussian random variables, the accuracy and speed of convergence of the proposed estimators can be improved by employing more accurate Riemann sums for the integrals and more accurate partial sums of the power series in the computation of the matching function than currently used.