On the sub-Gaussianity of the Beta and Dirichlet distributions

TL;DR

This paper determines the optimal sub-Gaussian proxy variance for Beta and Dirichlet distributions, providing new proofs and extending results to these distributions, with implications for inequalities in probability theory.

Contribution

It establishes the first known optimal proxy variance for the Dirichlet distribution and offers novel proof techniques for Beta distribution sub-Gaussianity.

Findings

Optimal proxy variance for Beta distribution proved

Derived the first proxy variance for Dirichlet distribution

Connected proxy variance to log-Sobolev and transport inequalities

Abstract

We obtain the optimal proxy variance for the sub-Gaussianity of Beta distribution, thus proving upper bounds recently conjectured by Elder (2016). We provide different proof techniques for the symmetrical (around its mean) case and the non-symmetrical case. The technique in the latter case relies on studying the ordinary differential equation satisfied by the Beta moment-generating function known as the confluent hypergeometric function. As a consequence, we derive the optimal proxy variance for the Dirichlet distribution, which is apparently a novel result. We also provide a new proof of the optimal proxy variance for the Bernoulli distribution, and discuss in this context the proxy variance relation to log-Sobolev inequalities and transport inequalities.

Figures

Equations

$$\mathbb{E}[\exp (\lambda (X-\mu ))]\le \exp \left(\frac{{\lambda }^2{\sigma }^2}{2}\right)\text{for all }\lambda \in \mathbb{R}.$$

$${\sigma }_{\text{opt}}^2(X)=\min \{{\sigma }^2\ge 0\text{ such that }X\text{ is }{\sigma }^2\text{-sub-Gaussian}\}.$$

$$\forall \lambda \in \mathbb{R},\mathbb{E}\left[{\mathrm{e}}^{\lambda (X-\mu )}\right]\le {\mathrm{e}}^{\frac{{\lambda }^2}{8}},$$

$$g(\mu )=\frac{1}{1-2\mu }\ln \frac{1-\mu }{\mu }$$

$$\mathbb{E}[\exp (\lambda (X-\mu ))]\le \exp \left(\frac{{\lambda }^2}{4g(\mu )}\right)\text{for all }\lambda \in \mathbb{R},$$

$$W(P,Q)=\underset{\pi \in \Pi (P,Q)}{\inf }{\int }_{\mathcal{X}\times \mathcal{X}}d(x,y)\pi (\mathrm{d}x,\mathrm{d}y),$$

$$W(P,Q)\le \sqrt{2cD(Q||P)},$$

$$\Vert P-Q{\Vert }_{\text{TV}}\le \sqrt{\frac{1}{g({\mu }_P)}D(Q||P)},$$

$$f(x)=\frac{1}{B(\alpha ,\beta )}{x}^{\alpha -1}(1-x{)}^{\beta -1},$$

$$\mathbb{E}[\exp (\lambda X)]={}_1{F}_1(\alpha ;\alpha +\beta ;\lambda )=\sum _{j=0}^{\infty }\frac{\Gamma (\alpha +j)\Gamma (\alpha +\beta )}{(j!)\Gamma (\alpha )\Gamma (\alpha +\beta +j)}{\lambda }^j.$$

$$\mathbb{E}[X^j]=\frac{(\alpha {)}_j}{(\alpha +\beta {)}_j},$$

$$\mathbb{E}[X]=\frac{\alpha }{\alpha +\beta },\mathrm{Var}[X]=\frac{\alpha \beta }{(\alpha +\beta {)}^2(\alpha +\beta +1)}.$$

$$\left\{\begin{array}[]{ll}\sigma_{\text{opt}}^{2}(\alpha,\beta)=\frac{\alpha}{(\alpha+\beta)x_{0}}\left(\frac{{}_{1}F_{1}(\alpha+1;\alpha+\beta+1;x_{0})}{{}_{1}F_{1}(\alpha;\alpha+\beta;x_{0})}-1\right)\cr\quad\quad\text{ where }x_{0}\text{ is the unique solution of the equation}\cr\ln(_{1}F_{1}(\alpha;\alpha+\beta;x_{0}))=\frac{\alpha x_{0}}{2(\alpha+\beta)}\left(1+\frac{{}_{1}F_{1}(\alpha+1;\alpha+\beta+1;x_{0})}{{}_{1}F_{1}(\alpha;\alpha+\beta;x_{0})}\right).\end{array}\right.$$

$$\mathbb{E}[\exp (\lambda (X-\mathbb{E}[X]))]$$

$$\exp \left(\frac{{\lambda }^2{\sigma }_0^2(\alpha )}{2}\right)$$

$$\mathbb{E}\left[{\left(X-\frac{1}{2}\right)}^{2j}\right]\frac{1}{(2j)!}\le \frac{{\sigma }_0^{2j}(\alpha )}{2^j}\frac{1}{j!}.$$

$$\mathbb{E}\left[{\left(X-\frac{1}{2}\right)}^{2j}\right]=\frac{(2j)!}{2^{2j}j!}\frac{\Gamma (2\alpha )\Gamma (\alpha +j)}{\Gamma (\alpha )\Gamma (2(\alpha +j))}=\frac{(2j)!}{2^{2j}j!}\frac{(\alpha {)}_j}{(2\alpha {)}_{2j}}.$$

$$\frac{(\alpha {)}_j}{(2\alpha {)}_{2j}}=\frac{1}{2^j\underset{l=1}{\prod ^j}(2\alpha +2l-1)}\le \frac{1}{(2(2\alpha +1){)}^j},$$

$${\sigma }_t^2=\frac{(1-t)}{4(1+\alpha +\beta )}+t\frac{\alpha \beta }{(\alpha +\beta {)}^2(\alpha +\beta +1)}=\frac{1}{4(1+\alpha +\beta )}+\frac{(\beta -\alpha {)}^2}{4(\alpha +\beta {)}^2(1+\alpha +\beta )}t.$$

$$u_t(x)\stackrel{\text{def}}{=}\exp \left(\frac{\alpha }{\alpha +\beta }x+\frac{{\sigma }_t^2}{2}{x}^2\right)-\mathbb{E}[\exp (xX)],\forall x\in \mathbb{R},$$

$$\left\{\begin{array}[]{ll}xu_{t}^{\prime\prime}(x)+(\alpha+\beta-x)u_{t}^{\prime}(x)-\alpha u_{t}(x)\cr\quad\quad\quad\quad\quad\quad\quad\quad=\frac{x}{16(\alpha+\beta)^{4}(1+\alpha+\beta)^{2}}\exp\left(\frac{\alpha}{\alpha+\beta}x+\frac{\sigma_{t}^{2}}{2}x^{2}\right)P_{2}(x;t),\cr u_{t}(0)=0\text{ and }u_{t}^{\prime}(0)=0,\end{array}\right.$$

$$\begin{gathered} P_2(x;t)=4(1-t)({\alpha }^2-{\beta }^2{)}^2(1+\alpha +\beta {)}^2 \\ -4({\beta }^2-{\alpha }^2)(1+\alpha +\beta )\left((\alpha +\beta {)}^2-t(\beta -\alpha {)}^2\right)x+{\left((\alpha +\beta {)}^2-t(\beta -\alpha {)}^2\right)}^2{x}^2. \end{gathered}$$

$$\begin{gathered} v_t(x)=16(\alpha +\beta {)}^4(1+\alpha +\beta {)}^2{u}_t(x)\exp \left(-\frac{\alpha }{\alpha +\beta }x-\frac{{\sigma }_t^2}{2}{x}^2\right), \\ =16(\alpha +\beta {)}^4(1+\alpha +\beta {)}^2\left(1-\mathbb{E}[\exp (xX)]\exp \left(-\frac{\alpha }{\alpha +\beta }x-\frac{{\sigma }_t^2}{2}{x}^2\right)\right). \end{gathered}$$

$$\displaystyle\left\{\begin{array}[]{ll}xv_{t}^{\prime\prime}(x)+Q_{1}(x;t)v_{t}^{\prime}(x)+Q_{0}(x;t)v_{t}(x)=xP_{2}(x;t),\cr v_{t}(0)=0\text{ and }v_{t}^{\prime}(0)=0,\end{array}\right.$$

$$Q_1(x;t)$$

$$Q_0(x;t)$$

$$v_t^{\prime \prime }(0)=\frac{P_2(0;t)}{1+Q_1(0;t)}=4({\beta }^2-{\alpha }^2{)}^2(1+\alpha +\beta )(1-t).$$

$${\Delta }_t=16t(1+\alpha +\beta {)}^2({\beta }^2-{\alpha }^2{)}^2{\left((\alpha +\beta {)}^2-t(\beta -\alpha {)}^2\right)}^2.$$

$$\forall \lambda \ge 0,\mathbb{E}[\exp (\lambda X)]={}_1{F}_1(\alpha ;\alpha +\beta ;\lambda )\le \exp \left(\frac{\lambda \alpha }{\alpha +\beta }+\frac{{\lambda }^2{\sigma }_t^2}{2}\right).$$

$$\begin{gathered} \mathbb{E}[\exp (\lambda X)]=\exp (\lambda )\mathbb{E}[\exp (-\lambda (1-X))] \\ \le \exp (\lambda )\exp \left(-\lambda (1-\frac{\alpha }{\alpha +\beta })+\frac{{\sigma }_t^2{\lambda }^2}{2}\right)=\exp \left(\lambda \frac{\alpha }{\alpha +\beta }+\frac{{\sigma }_t^2{\lambda }^2}{2}\right). \end{gathered}$$

$$M=\{0<x<x_0\text{ such that }{u}_0^{\prime }(x)=0\text{ and }{u}_0^{\prime }\text{ changes sign at }x\}.$$

Full text

1 Introduction

The sub-Gaussian property (Buldygin and Kozachenko,, 1980, 2000; Pisier,, 2016) and related concentration inequalities (Boucheron et al.,, 2013; Raginsky and Sason,, 2013) have attracted a lot of attention in the last couple of decades due to their applications in various areas such as pure mathematics, physics, information theory and computer sciences. Recent interest focused on deriving the optimal proxy variance for discrete random variables like the Bernoulli distribution (Buldygin and Moskvichova,, 2013; Kearns and Saul,, 1998; Berend and Kontorovich,, 2013) and the missing mass (McAllester and Schapire,, 2000; McAllester and Ortiz,, 2003; Berend and Kontorovich,, 2013; Ben-Hamou et al.,, 2017). Our focus is instead on two continuous random variables, the Beta and Dirichlet distributions, for which the optimal proxy variance was not known to the best of our knowledge. Some upper bounds were recently conjectured by Elder, (2016) that we prove in the present article by providing the optimal proxy variance for both Beta and Dirichlet distributions. Similar concentration properties of the Beta distribution have been recently used in many contexts including Bayesian adaptive data analysis (Elder,, 2016), Bayesian nonparametrics (Castillo,, 2016) and spectral properties of random matrices (Perry et al.,, 2016).

We start by reminding the definition of sub-Gaussian property for random variables:

Definition 1** (Sub-Gaussian variables).**

A random variable $X$ with finite mean $\mu=\mathbb{E}[X]$ is sub-Gaussian if there is a positive number $\sigma$ such that:

[TABLE]

Such a constant $\sigma^{2}$ is called a proxy variance (or sub-Gaussian norm), and we say that $X$ is $\sigma^{2}$-sub-Gaussian. If $X$ is sub-Gaussian, one is usually interested in the optimal proxy variance:

[TABLE]

Note that the variance always gives a lower bound on the optimal proxy variance: $\text{Var}[X]\leq\sigma_{\text{opt}}^{2}(X)$. In particular, when $\sigma_{\text{opt}}^{2}(X)=\text{Var}[X]$, $X$ is said to be strictly sub-Gaussian.

Every compactly supported distribution, as is the Beta$(\alpha,\beta)$ distribution, is sub-Gaussian. This can be seen by Hoeffding’s classic inequality: any random variable $X$ supported on $[0,1]$ with mean $\mu$ satisfies

[TABLE]

thus exhibiting $\frac{1}{4}$ as an upper bound to the proxy variance. This bound can be improved by taking into account the location of the mean $\mu$ within the interval $[0,1]$. An early step in this direction is the second inequality in Hoeffding, (1963) paper, indexed (2.2). It states that if $\mu<1/2$, then for any positive $\epsilon$, $\mathbb{P}(X-\mu>\epsilon)\leq\mathrm{e}^{-\epsilon^{2}g(\mu)}$, where

[TABLE]

thus indicating that $X$ has a right tail lighter than a Gaussian tail of variance $\frac{1}{2g(\mu)}$. Hoeffding’s result was strengthened by Kearns and Saul, (1998) to comply with Definition 1 of sub-Gaussianity111Note indeed that Equation (1), together with Markov inequality, imply $\mathbb{P}(X-\mu>\epsilon)\leq\mathrm{e}^{-\frac{\epsilon^{2}}{2\sigma^{2}}}$. as follows

[TABLE]

thus indicating that $\frac{1}{2g(\mu)}$ is a distribution-sensitive proxy variance for any $[0,1]$-supported random variable with mean $\mu$ (see also Berend and Kontorovich,, 2013, for a detailed proof of this result). If this is the optimal proxy variance for the Bernoulli distribution (see Theorem 2.1 and Theorem 3.1 of Buldygin and Moskvichova,, 2013), it is clear from our result that it does not hold true for the Beta distribution. However, fixing $\frac{\alpha}{\alpha+\beta}=\mu$ and letting $\alpha\to 0$, $\beta\to 0$, the $\operatorname{Beta}(\alpha,\beta)$ distribution concentrates to the $\operatorname{Bern}(\mu)$ distribution, and we show that we recover the optimal proxy variance for the Bernoulli distribution (Theorem 2).

An interesting common feature between optimal proxy variances for the Bernoulli distribution: $\frac{1}{2g(\mu)}$, and that of the Beta distribution derived later on, is that they deteriorate in a similar fashion as the mean $\mu$ goes to 0 or 1, see for instance the left panel of Figure 1. We briefly present here classical proof techniques for sub-Gaussianity hinging on certain tools from functional analysis. We show how they apply in the Bernoulli setting, and let as an interesting open problem how our proof in the Beta distribution setting could be supplemented by these same functional analysis tools.

Essentially two (related) functional inequalities allow one to derive a sub-Gaussian property: log-Sobolev inequalities, which date back to Gross, (1975), and transport inequalities. The relation with the former inequalities is called Herbst’s argument. It states that if a probability measure satisfies a log-Sobolev inequality with some constant, then it is sub-Gaussian with the same constant as a proxy variance222The implied predicate is actually stronger than sub-Gaussianity, but it is not useful for our purposes. (see for instance Ledoux,, 1999, Section 2.3 and Proposition 2.3). The optimal constant in the log-Sobolev inequality satisfied by the Bernoulli distribution also produces its optimal proxy variance (Ledoux,, 1999, Corollary 5.9).

The relation with transport inequalities is usually referred to as Marton’s argument (see for instance Raginsky and Sason,, 2013, Section 3.4). Define the Wasserstein distance between two probability measures $P$ and $Q$ on a space $\mathcal{X}$ by

[TABLE]

where $\Pi(P,Q)$ is the set of probability measures on $\mathcal{X}\times\mathcal{X}$ with fixed marginal distributions respectively $P$ and $Q$. The Wasserstein distance depends on some choice of a distance $d$ on $\mathcal{X}$. A probability measure $P$ is said to satisfy a transport inequality with constant $c$, if for any probability measure $Q$ dominated by $P$,

[TABLE]

where $D(Q||P)$ is the entropy, or Kullback–Leibler divergence, between $P$ and $Q$. The transport inequality (4) is denoted by $\mathsf{T}(c)$.

Bobkov and Götze, (1999) proved that $\mathsf{T}(c)$ implies $c$-sub-Gaussianity. See also Proposition 3.6 and Theorem 3.4.4 of Raginsky and Sason, (2013) for general results. Further developments in the discrete $\mathcal{X}$ setting are interesting for our purposes. Equip a discrete space $\mathcal{X}$ with the Hamming metric, $d(x,y)=\mathbb{1}_{\{x\neq y\}}$. The induced Wasserstein distance then reduces to the total variation distance, $W(P,Q)=\|P-Q\|_{\text{TV}}$. In that setting, Ordentlich and Weinberger, (2005) proved the distribution-sensitive transport inequality:

[TABLE]

where the function $g$ is defined in Equation (2) and the coefficient $\mu_{P}$ is called the balance coefficient of $P$, and is defined by $\mu_{P}=\underset{A\subset\mathcal{X}}{\max}\min\{P(A),1-P(A)\}$. In particular, the Bernoulli balance coefficient is easily shown to coincide with its mean. Hence, applying the result of Bobkov and Götze, (1999) to the $\mathsf{T}\left(\frac{1}{2g(\mu_{P})}\right)$ transport inequality (5) yields a distribution-sensitive proxy variance of $\frac{1}{2g(\mu)}$ for the Bernoulli with mean $\mu$. It is optimal, see for instance Theorem 3.4.6 of Raginsky and Sason, (2013). This viewpoint highlights the key role played by the balance coefficient in the non-uniformity of the optimal proxy variance for discrete distributions such as the Bernoulli. However, it is not clear how this argument would carry over to non discrete distributions such as the Beta distribution for explaining similar sensitivity to the mean. However, to quote Raginsky and Sason, (2013), the general approach may not produce optimal concentration estimates, that often require case-by-case treatments. This is the route followed in this note for the Beta distribution.

The outline of the note is as follows. We introduce the Beta distribution and state the main result (Theorem 1) in Section 2.1. We then prove our result depending on whether $\alpha=\beta$ (Section 2.2) or $\alpha\neq\beta$ (Section 2.3). In the first case, the proof is elementary and based on comparing the coefficients of the entire series representations of the functions of both sides of inequality (1). However, it does not directly carry over to the second case, whose proof requires some finer analysis tool: the study of the ordinary differential equation (ODE) satisfied by the confluent hypergeometric function ${}_{1}F_{1}$. Although the second proof also covers the case $\alpha=\beta$ upon slight modifications, the independent proof for the symmetric case is kept owing to its simplicity. As a by-product, we derive the optimal proxy variance for the Bernoulli and the Dirichlet distributions in Section 3. The R code for the plots presented in this note and for a function deriving the optimal proxy variance in terms of $\alpha$ and $\beta$ is available at http://www.julyanarbel.com/software.

2 Optimal proxy variance for the Beta distribution

2.1 Notations and main result

The $\operatorname{Beta}(\alpha,\beta)$ distribution, with $\alpha,\beta>0$, is characterized by a density on the segment $[0,1]$ given by:

[TABLE]

where $B(\alpha,\beta)=\int_{0}^{\infty}x^{\alpha-1}(1-x)^{\beta-1}\mathrm{d}x=\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$ is the Beta function. The moment-generating function of a $\text{Beta}(\alpha,\beta)$ distribution is given by a confluent hypergeometric function (also known as Kummer’s function):

[TABLE]

This is equivalent to say that the $j^{\text{th}}$ raw moment of a $\operatorname{Beta}(\alpha,\beta)$ random variable $X$ is given by:

[TABLE]

where $(x)_{j}=x(x+1)\cdots(x+j-1)=\frac{\Gamma(x+j)}{\Gamma(x)}$ is the Pochhammer symbol, also known in the literature as a rising factorial. In particular, the mean and variance are given by:

[TABLE]

The Beta distribution is ubiquitous in statistics. It plays a central role in the binomial model in Bayesian statistics where it is a conjugate prior distribution (the associated posterior distribution is also Beta): if $X\sim\text{Binomial}(\theta,N)$ and $\theta\sim\operatorname{Beta}(\alpha,\beta)$, then $\theta|X\sim\operatorname{Beta}(\alpha+X,\beta+N-X)$. It is also key to Bayesian nonparametrics where it embodies, among others, the distribution of the breaks in the stick-breaking representation of the Dirichlet process and the Pitman–Yor process; marginal distributions of Polya trees (Castillo,, 2016); the posterior distribution of discovery probabilities under a Bayesian nonparametrics model (Arbel et al.,, 2017). Our main result opens new research avenues for instance about asymptotic (frequentist) assessments of these procedures.

Our main result regarding the Beta distribution is the following:

Theorem 1** (Optimal proxy variance for the Beta distribution).**

For any $\alpha,\beta>0$, the Beta distribution $\operatorname{Beta}(\alpha,\beta)$ is $\sigma_{\text{opt}}^{2}(\alpha,\beta)$-sub-Gaussian with optimal proxy variance $\sigma_{\text{opt}}^{2}(\alpha,\beta)$ given by:

[TABLE]

*A simple and explicit upper bound to $\sigma_{\text{opt}}^{2}(\alpha,\beta)$ is given by $\sigma_{0}^{2}(\alpha,\beta)=\frac{1}{4(\alpha+\beta+1)}$:

• for $\alpha\neq\beta$ we have $\text{{Var}}[\operatorname{Beta}(\alpha,\beta)]<\sigma_{\text{opt}}^{2}(\alpha,\beta)<\frac{1}{4(\alpha+\beta+1)}$

• for $\alpha=\beta$ we have $\text{{Var}}[\operatorname{Beta}(\alpha,\alpha)]=\sigma_{\text{opt}}^{2}(\alpha,\alpha)=\frac{1}{4(2\alpha+1)}$.*

Equation (8) defining $x_{0}$ is a transcendental equation, the solution of which is not available in closed form. However, it is simple to evaluate numerically. The values of the variance, optimal proxy variance and its simple upper bound are illustrated on Figure 1. Note that for a fixed value of the sum of the parameters, $\alpha+\beta=S$, the optimal proxy variance deteriorates when $\alpha$, or equivalently $\beta$, gets close to 0 or to $S$. This is reminiscent of the Bernoulli optimal proxy variance behavior which deteriorates when the success probability moves away from $\frac{1}{2}$ (Buldygin and Moskvichova,, 2013).

The intuition of the proof can be seen from Figure 2 (Section 2.3.3) where we represent the difference $\lambda\mapsto\exp\left(\mathbb{E}[X]\lambda+\frac{\sigma^{2}}{2}\lambda^{2}\right)-\mathbb{E}[\exp(\lambda X)]$ for various values of $\sigma^{2}$. The main argument is that the optimal proxy variance is obtained for the curve (in magenta) whose positive local minimum equals zero, thus leading to the system of equations of Theorem 1.

Corollary 1**.**

The Beta distribution $\operatorname{Beta}(\alpha,\beta)$ is strictly sub-Gaussian if and only if $\alpha=\beta$.

As a direct consequence, we obtain the strict sub-Gaussianity of the uniform, the arc-sine and the Wigner semicircle distributions, as special cases up to a trivial rescaling of the Beta$(\alpha,\alpha)$ distribution respectively with $\alpha$ equal to $1$, $\frac{1}{2}$ and $\frac{3}{2}$.

2.2 The $\text{Beta}(\alpha,\alpha)$ distribution is strictly sub-Gaussian

Let $\sigma_{0}^{2}(\alpha)=\text{Var}[\operatorname{Beta}(\alpha,\alpha)]=\frac{1}{4(2\alpha+1)}$. Since a random variable $X\sim\operatorname{Beta}(\alpha,\alpha)$ is symmetric around $\frac{1}{2}$, only its even centered moments are non-zero. The reason why \mathbb{E}[\exp(\lambda(X-\mathbb{E}[X]))]\leq\exp\big{(}\frac{\lambda^{2}\sigma_{0}^{2}(\alpha)}{2}\big{)} is because the coefficients of the series expansions at $\lambda=0$ of each side:

[TABLE]

satisfy the inequalities:

[TABLE]

Indeed, algebra yields:

[TABLE]

Combining the expression of the raw moments (7) with the following inequality:

[TABLE]

in (9) concludes the proof.

Remark 1**.**

The non-symmetrical distribution with $\alpha\neq\beta$ has even centered moments whose expressions are not as simple as (12). Moreover, it has obviously non-zero odd centered moments. For this last reason, the present proof does not carry over to the case $\alpha\neq\beta$.

2.3 Optimal proxy variance for the $\text{Beta}(\alpha,\beta)$ distribution

2.3.1 Connection with ordinary differential equations

In this section, we assume that $X\sim\text{Beta}(\alpha,\beta)$ with $\beta\neq\alpha$. We denote $\sigma_{0}^{2}=\frac{1}{4(\alpha+\beta+1)}$ (we omit the dependence on $\alpha$ and $\beta$ for compactness) and define for all $t\in\mathbb{R}$:

[TABLE]

In other words, the decreasing function $t\mapsto\sigma^{2}_{t}$ maps the interval $[0,1]$ to the interval $[\sigma_{1}^{2},\sigma_{0}^{2}]$ with $\sigma_{1}^{2}=\text{Var}[X]$. Then, we introduce the function $u_{t}$ defined by:

[TABLE]

where $\sigma^{2}_{t}$-sub-Gaussianity amounts to non negativity of $u_{t}$ on $\mathbb{R}$. Since the confluent hypergeometric function $y:x\mapsto y(x)=\,_{1}F_{1}(\alpha,\alpha+\beta;x)$ satisfies the linear second order ordinary differential equation $xy^{\prime\prime}(x)+(\alpha+\beta-x)y^{\prime}(x)-\alpha y(x)=0$, we obtain together with equation (6) that $u_{t}$ is the unique solution of the Cauchy problem:

[TABLE]

where $P_{2}$ is a polynomial of degree $2$ in $x$:

[TABLE]

For normalization purposes, we also define:

[TABLE]

The function $v_{t}$ is the unique solution of the Cauchy problem:

[TABLE]

with:

[TABLE]

Note that $u_{t}$ and $v_{t}$ have the same sign hence proving that $u_{t}$ is positive (resp. negative) is equivalent to proving that $v_{t}$ is positive (resp. negative). From standard theory on ODEs (Birkhoff and Rota,, 1989; Robinson,, 2004), we get that the functions $u_{t}$ and $v_{t}$ are $\mathcal{C}^{\infty}(\mathbb{R})$. Indeed, the only possible singularity is at $x=0$ but the initial conditions imply that the function is regular at this point. In particular, a Taylor expansion at $x=0$ shows that:

[TABLE]

We also observe that the discriminant of the polynomial $x\mapsto P_{2}(x;t)$ is given by:

[TABLE]

Hence we conclude that for $t>0$, $P_{2}$ admits two distinct real zeros that are positive, while for $t<0$ it remains strictly positive on $\mathbb{R}$. For $t=0$, $P_{2}$ admits a double zero and thus remains positive on $\mathbb{R}$ appart from its zero.

By definition (15), we want to study the sign of $v_{t}$ on $\mathbb{R}$. Indeed, showing that $v_{t}$ is positive on $\mathbb{R}$ then $X$ is equivalent to showing $\sigma_{t}^{2}$-sub-Gaussianity. We first observe that we may restrict the sign study on $\mathbb{R}_{+}$. Indeed, if we prove that:

[TABLE]

then, the case $\lambda<0$ is automatically obtained by noting that $1-X\sim\operatorname{Beta}(\beta,\alpha)$, whose mean is $\frac{\beta}{\alpha+\beta}=1-\frac{\alpha}{\alpha+\beta}$. Therefore, applying (20) to $1-X$ gives that for all $\lambda<0$:

[TABLE]

Eventually, in agreement with the general theory, we observe that for $t>1$ (i.e. $\sigma^{2}_{t}<\text{Var}[X]$), $X$ is not $\sigma_{t}^{2}$-sub-Gaussian. Indeed, the series expansion at $x=0$ (19), shows that for $t>1$, $v_{t}$ is strictly negative in a neighborhood of [math]. On the contrary, for $t<1$, the function $v_{t}$ is strictly positive in a neighborhood of [math] so that we may not directly conclude. Note also that for any value of $t$, we always have $\underset{x\to\infty}{\lim}v_{t}(x)=+\infty$.

2.3.2 Proof that the $\text{Beta}(\alpha,\beta)$ distribution is $\sigma_{0}^{2}$-sub-Gaussian

In this section, we take $t=0$. As explained above, this corresponds to a case where $P_{2}$ is positive on $\mathbb{R}$ (apart from its double zero). We prove that $u_{0}(x)>0$ for $x>0$ by proceeding by contradiction. Let us assume that there exists $x_{1}>0$ such that $u_{0}(x_{1})=0$. Since the non-empty set $\{x>0\,/\,u_{0}(x)=0\}$ is compact (because $u_{0}\in\mathcal{C}^{\infty}(\mathbb{R})$) and excludes a neighborhood of [math], we may define $x_{0}=\min\{x>0\text{ such that }u_{0}(x)=0\}>0$. Let us now define the set:

[TABLE]

Since $u_{0}(0)=u_{0}(x_{0})=0$ and the facts that $u_{0}$ is strictly positive in a neighborhood of [math] and $u_{0}^{\prime}$ is continuous on $\mathbb{R}$, Rolle’s theorem shows that $M$ is not empty and that:

[TABLE]

Evaluating the ODE (14) at $x=m$ and using the fact that the polynomial $P_{2}$ is positive on $\mathbb{R}$ (appart from its double zero) leads to:

[TABLE]

However, combined with $u_{0}^{\prime}(m)=0$, this contradicts the fact that $u_{0}^{\prime}$ changes sign at $x=m$.

Thus, we conclude that there cannot exist $x_{1}>0$ such that $u_{0}(x_{1})=0$. Since $u_{0}$ is strictly positive in a neighborhood of [math] and continuous on $\mathbb{R}$, we conclude that it must remain strictly positive on $\mathbb{R}_{+}^{*}$.

Remark 2**.**

In this proof, the case $\beta=\alpha$ requires an adaptation since $u_{0}^{\prime\prime}(0)=0$. Thus, we must determine $u_{0}^{(4)}(0)>0$ ($u_{0}^{(3)}(0)=0$ by symmetry) to ensure that the function $u_{0}$ is locally convex and remains strictly positive in a neighborhood of [math]. Apart from this minor verification, the rest of the proof applies also to this case.

2.3.3 Proof of the optimal proxy variance for the $\text{Beta}(\alpha,\beta)$ distribution

In this section we assume that $\beta\neq\alpha$. From general theorems regarding ODEs, we have that the application:

[TABLE]

is smooth ($g\in\mathcal{C}^{\infty}([0,+\infty)\times[0,+\infty))$). Indeed, the $t$-dependence of the coefficients of the ODE (18) is polynomial and thus smooth. The $x$-dependence of the coefficients of the ODE (18) is polynomial and as explained above, the only possible singularity in $x$ is at $x=0$ but initial conditions ensure that the solutions $x\mapsto v_{t}(x)$ are always regular there. Since for all $t\geq 0$ we have $\underset{x\to\infty}{\lim}v_{t}(x)=+\infty$, we also have that the function:

[TABLE]

is continuous.

We now observe that for any $0\leq t<1$, the functions $v_{t}$ are strictly positive in a neighborhood of [math]. More precisely, if we choose a segment $[0,t_{0}]$ with $t_{0}<1$, then for all $0\leq t\leq t_{0}$ we have that $v_{t}^{\prime\prime}(0)\geq v_{t_{0}}^{\prime\prime}(0)=4(\beta^{2}-\alpha^{2})^{2}(1+\alpha+\beta)(1-t_{0})>0$. Hence, we may choose $\eta>0$ such that for all $0\leq t\leq t_{0}$, we have $v_{t}$ is strictly positive on $]0,\eta]$. Moreover, since $\underset{x\to\infty}{\lim}v_{0}(x)=+\infty$, $v_{0}$ is bounded from below on $[\eta,+\infty[$ by a constant $A>0$. Thus, since $g$ is continuous, there exists a neighborhood of $t=0$ in which all solutions $v_{t}$ remain greater than $\frac{A}{2}$ on $[\eta,+\infty[$ and thus strictly positive on $\mathbb{R}_{+}^{*}$. This shows that for $t>0$, $\sigma_{0}^{2}$ is not optimal.

Let us now introduce the set:

[TABLE]

Then, from the results presented above, we know that $T_{+}$ is non-empty, that it contains a neighborhood of [math] and that it is bounded from above by $1$. Moreover, by connection with the initial problem (15), $T_{+}$ is an interval and thus is of the form $[0,t_{\text{opt}}]$ with $0<t_{\text{opt}}<1$. Indeed $t\in T_{+}$ implies by construction that for all $s\leq t$, $s\in T_{+}$. Note also that $t<1$ since $v_{1}$ is strictly negative in a neighborhood of [math] and thus $\min\{v_{1}(x),x\in\mathbb{R}_{+}^{*}\}<0$. Hence, the continuity of $h$ shows that there exists a neighborhood of $t=1$ in which the solutions $v_{t}$ are non-positive on $\mathbb{R}_{+}^{*}$. For $t=t_{\text{opt}}$ the function $v_{t_{\text{opt}}}$ must have a zero on $\mathbb{R}_{+}^{*}$ otherwise by continuity of $h$ we may find a neighborhood of $t_{\text{opt}}$ for which $\min\{v_{t}(x),\,x\in\mathbb{R}_{+}^{*}\}$ remains strictly positive thus contradicting the maximality of $t_{\text{opt}}$. Since $v_{t_{\text{opt}}}$ must remain positive, the zero is at least a double zero and therefore we find that there exists $x_{0}>0$ such that $v_{t_{\text{opt}}}(x_{0})=0$, $v^{\prime}_{t_{\text{opt}}}(x_{0})=0$ and $v^{\prime\prime}_{t_{\text{opt}}}(x_{0})\geq 0$. From (6) and (15), the conditions $v_{t_{\text{opt}}}(x_{0})=0$, $v^{\prime}_{t_{\text{opt}}}(x_{0})=0$ are equivalent to the following system of equations (we use here the contiguous relations for the confluent hypergeometric function: ${}_{1}F_{1}^{\prime}(a;b;x)=\frac{a}{b}~{}_{1}F_{1}(a;b;x)$):

[TABLE]

This is equivalent to say that $x_{0}\equiv x_{0}(\alpha,\beta)$ is the solution of the transcendental equation:

[TABLE]

and that $\sigma^{2}_{t_{\text{opt}}}$ is given by:

[TABLE]

Note that by symmetry, we have $x_{0}(\beta,\alpha)=-x_{0}(\alpha,\beta)$ hence, $\sigma^{2}_{t_{\text{opt}}}(\beta,\alpha)=\sigma^{2}_{t_{\text{opt}}}(\alpha,\beta)$. Moreover, if $\beta>\alpha$ then $x_{0}(\alpha,\beta)>0$ while $\alpha>\beta$ implies $x_{0}(\alpha,\beta)<0$. We may illustrate the situation with Figure 2 which displays the difference function $x\mapsto u_{t}(x)$.

Remark 3**.**

The system of equations (23) admits only one solution on $\mathbb{R}_{+}^{*}$. Indeed, let us transpose the problem from $v_{t_{\text{opt}}}$ to $u_{t_{\text{opt}}}$ using (15) and assume that there exist two points $0<x_{0}<x_{1}$ such that $u_{t_{\text{opt}}}(x_{0})=0$, $u^{\prime}_{t_{\text{opt}}}(x_{0})=0$ and $u_{t_{\text{opt}}}(x_{1})=0$, $u^{\prime}_{t_{\text{opt}}}(x_{1})=0$ with $u_{t_{\text{opt}}}$ strictly positive on $(x_{0},x_{1})$ (hence $u^{\prime\prime}_{t_{\text{opt}}}(x_{0})\geq 0$ and $u^{\prime\prime}_{t_{\text{opt}}}(x_{1})\geq 0$). Using (14), this implies that $P_{2}(x_{0};t_{\text{opt}})\geq 0$ and $P_{2}(x_{1};t_{\text{opt}})\geq 0$. If we denote $x_{-}<x_{+}$ the potential distinct positive zeros of $x\mapsto P_{2}(x;t_{\text{opt}})$ we may exclude that $x_{0}\leq x_{-}$. Indeed, if $x_{0}\leq x_{-}$ then we may apply the same argument to $u_{t_{\text{opt}}}$ on the interval $[0,x_{0}]$ as the one developed for $u_{0}$ in Section 2.3.2 and obtain a contradiction. Thus, the only remaining case is to assume $x_{1}>x_{0}>x_{+}$. In that case, since $u_{t_{\text{opt}}}(x_{0})=0$, $u^{\prime}_{t_{\text{opt}}}(x_{0})=0$, $u_{t_{\text{opt}}}(x_{1})=0$ and $x\mapsto P_{2}(x;t_{\text{opt}})$ is positive on $[x_{0},x_{1}]$, we may apply the same argument to $u_{t_{\text{opt}}}$ on the interval $[x_{0},x_{1}]$ as the one developed for $u_{0}$ in Section 2.3.2 and obtain a contradiction.

3 Relations to other distributions

3.1 Optimal proxy variance for the Bernoulli distribution

We show that our proof technique can be used to recover the optimal proxy variance for the Bernoulli distribution, known since Kearns and Saul, (1998). This is illustrated by the center panel of Figure 1.

Theorem 2** (Optimal proxy variance for the Bernoulli distribution).**

For any $\mu\in(0,1)$, the Bernoulli distribution with mean $\mu$ is sub-Gaussian with optimal proxy variance $\sigma_{\text{opt}}^{2}(\mu)$ given by:

[TABLE]

Proof.

In the limit $\alpha\to 0$ with $\frac{\alpha}{\alpha+\beta}$ fixed equal to $\mu$, the differential equation (14) simplifies into:

[TABLE]

with the Cauchy initial conditions $u_{t,\mu}(0)=0$ and $u^{\prime}_{t,\mu}(0)=0$. The solution of this Cauchy problem is explicit and given by:

[TABLE]

Therefore the optimal proxy variance is given by $\sigma_{\text{opt}}^{2}(\mu)=\frac{1}{4}-\frac{1}{4}x(2\mu-1)^{2}t_{0}$ where $t_{0}$ is determined by the system of equations: $u_{t_{0},\mu}(x_{0})=0$ and $u^{\prime}_{t_{0},\mu}(x_{0})$, thus defining implicitly $t_{0}$ and $x_{0}$ as functions of $\mu$. In order to solve explicitly the last system of equations, we perform the change of variables: $(\mu,t)=\left(\frac{s}{s+1},\frac{2\tilde{t}}{s+1}+1\right)$ so that the solution (25) is now given by:

[TABLE]

Consequently, we have to solve the system:

[TABLE]

Introducing another change of variable $(x_{0},y_{0})=(x_{0},x_{0}\tilde{t}_{0})$, the last system is equivalent to:

[TABLE]

We now observe that $y_{0}=s-1$ and $x_{0}=-2\ln s$ is a solution of the former system. Performing back the various changes of variables, this is equivalent to say that $\tilde{t}_{0}=\frac{y_{0}}{x_{0}}=\frac{2(1-s)}{\ln s}$ so that $t_{0}=\frac{4(1-s)}{(s+1)\ln s}+1$ or equivalently $t_{0}=\frac{1}{(2\mu-1)^{2}}+\frac{2}{(2\mu-1)\ln\frac{1-\mu}{\mu}}$. Consequently, the optimal proxy variance is given by:

[TABLE]

which is precisely the optimal proxy variance of a Bernoulli random variable with mean $\mu$. ∎

3.2 Optimal proxy variance for the Dirichlet distribution

We start by reminding the definition of sub-Gaussian property for random vectors:

Definition 2** (Sub-Gaussian vectors).**

A random $d$-dimensional vector $\boldsymbol{X}$ with finite mean $\boldsymbol{\mu}=\mathbb{E}[\boldsymbol{X}]$ is $\sigma^{2}$-sub-Gaussian if the random variable $\boldsymbol{u}^{\top}\,\boldsymbol{X}$ is $\sigma^{2}$-sub-Gaussian for any unit vector $\boldsymbol{u}$ in the simplex $\mathcal{S}^{d-1}=\{\boldsymbol{u}\in[0,1]^{d}\,/\,\underset{i=1}{\overset{d}{\sum}}u_{i}=1\}$. This is equivalent to say that:

[TABLE]

where $\|\boldsymbol{\lambda}\|^{2}=\underset{i=1}{\overset{d}{\sum}}\lambda_{i}^{2}$. Eventually, a random vector $\boldsymbol{X}$ is said to be strictly sub-Gaussian, if the random variables $\boldsymbol{u}^{\top}\,\boldsymbol{X}$ are strictly sub-Gaussian for any unit vector $\boldsymbol{u}\in\mathcal{S}^{d-1}$.

Let $d\geq 2$. The Dirichlet distribution $\operatorname{Dir}(\boldsymbol{\alpha})$, with positive parameters $\boldsymbol{\alpha}=(\alpha_{1},\ldots,\alpha_{d})^{\top}$, is characterized by a density on the simplex $\mathcal{S}^{d-1}$ given by:

[TABLE]

where $B(\boldsymbol{\alpha})=\frac{1}{\Gamma(\bar{\boldsymbol{\alpha}})}\underset{i=1}{\overset{d}{\prod}}\Gamma(\alpha_{i})$ and $\bar{\boldsymbol{\alpha}}=\underset{i=1}{\overset{d}{\sum}}\alpha_{i}$. It generalizes the Beta distribution in the sense that the components are Beta distributed. More precisely, for any non-empty and strict subset $\mathcal{I}$ of $\{1,\ldots,d\}$:

[TABLE]

However, we remind the reader that the components $(X_{i})_{1\leq i\leq d}$ are not independent and the variance/covariance matrix is given by:

[TABLE]

Eventually, if we define $\boldsymbol{n}=(n_{1},\dots,n_{d})^{\top}\in\mathbb{N}^{d}$, then the moments of the Dirichlet distribution are given by:

[TABLE]

This is equivalent to say that the moment-generating function of the Dirichlet distribution is:

[TABLE]

where we have defined $\boldsymbol{\lambda}=(\lambda_{1},\dots,\lambda_{d})^{\top}\in\mathbb{R}^{d}$ and $\bar{\boldsymbol{n}}=\underset{i=1}{\overset{d}{\sum}}n_{i}.$

Let us define $\boldsymbol{e}_{i}$ the $i^{\text{th}}$ canonical vector of $\mathbb{R}^{d}$ and $\boldsymbol{X}=(X_{1},\dots,X_{d})^{\top}\sim\operatorname{Dir}(\boldsymbol{\alpha})$. From Definition 1 and the results regarding the $\text{Beta}(\alpha,\beta)$ distribution obtained in Section 2.3, we immediately get that $\boldsymbol{e}_{i}^{\top}\,\boldsymbol{X}=X_{i}$ is $\sigma_{i}^{2}$-sub-Gaussian with $\sigma_{i}^{2}\overset{\text{def}}{=}\sigma_{\text{opt}}^{2}(\alpha_{i},\bar{\boldsymbol{\alpha}}-\alpha_{i})$ defined from Theorem 1. Moreover, in direction $\boldsymbol{e}_{i}$, $\sigma_{i}^{2}$ is the optimal proxy variance. Therefore, the remaining issue is to generalize these results for arbitrary unit vectors on $\mathcal{S}^{d-1}$. We obtain the following result:

Theorem 3** (Optimal proxy variance for the Dirichlet distribution).**

For any parameter $\boldsymbol{\alpha}$, the Dirichlet distribution $\operatorname{Dir}(\boldsymbol{\alpha})$ is sub-Gaussian with optimal proxy variance $\sigma_{\text{opt}}^{2}(\boldsymbol{\alpha})$ given from Theorem 1 and:

[TABLE]

Proof.

We first observe that the computations of $\sigma^{2}_{\text{opt}}(\alpha_{i},\beta_{i}=\bar{\boldsymbol{\alpha}}-\alpha_{i})$ correspond to cases where the sum $\alpha_{i}+\beta_{i}$ is fixed to $\bar{\boldsymbol{\alpha}}$ and thus independent of $i$. Therefore, $\sigma^{2}_{\text{opt}}(\alpha_{i},\bar{\boldsymbol{\alpha}}-\alpha_{i})$ is maximal when $|(\bar{\boldsymbol{\alpha}}-\alpha_{i})-\alpha_{i}|$ is minimal, i.e. when the distance from $\alpha_{i}$ to $\frac{1}{2}\bar{\boldsymbol{\alpha}}$ is minimal. It is easy to see that this corresponds to choosing $\alpha_{i}=\alpha_{\text{max}}=\max\{\alpha_{i}\,\,,\,\,1\leq i\leq d\}$ (by looking at the two possible cases $\alpha_{\text{max}}\leq\frac{1}{2}\bar{\boldsymbol{\alpha}}$ and $\alpha_{\text{max}}>\frac{1}{2}\bar{\boldsymbol{\alpha}}$).

We then observe that $\sigma^{2}_{\text{max}}$ cannot be improved. Indeed, let us denote $i_{0}$ one of the components for which the maximum is obtained. Then, if we take $\boldsymbol{u}=\boldsymbol{e}_{i_{0}}$, the discussion presented above shows that $\sigma_{i_{0}}^{2}=\sigma^{2}_{\text{max}}$ is the optimal proxy variance in this direction. Hence the optimal proxy variance cannot be lower than $\sigma^{2}_{\text{max}}$.

Let us now prove that $\boldsymbol{X}$ is $\sigma^{2}_{\text{max}}$-sub-Gaussian. Let $\boldsymbol{u}=(u_{1},\dots,u_{d})^{\top}$ be a unit vector on $\mathcal{S}^{d-1}$ and $\lambda\in\mathbb{R}$. We define for clarity $\boldsymbol{\lambda}=\lambda\boldsymbol{u}$. We have:

[TABLE]

Note that we also have:

[TABLE]

Moreover we have the inequality:

[TABLE]

because both sides have the same number of terms in the product (i.e. $\bar{\boldsymbol{n}}$) but those of the right hand side are always greater or equal to those of the left hand side. Hence, from (26) and (27), we find:

[TABLE]

Using the optimal proxy variance of the Beta distribution proven in Theorem 1, we find:

[TABLE]

thus showing that $\boldsymbol{X}$ is $\sigma^{2}_{\text{max}}$-sub-Gaussian and concluding the proof. ∎

Note that using Theorem 3, we obtain the following corollary:

Corollary 2**.**

For any integer $d\geq 2$, the Dirichlet distribution $\operatorname{Dir}(\alpha_{1},\dots,\alpha_{d})$ is strictly sub-Gaussian if and only if $d=2$ and $\alpha_{1}=\alpha_{2}$.

Indeed, we first need to require $\alpha_{1}=\dots=\alpha_{d}\overset{\text{def}}{=}\alpha$ so that all directions have the same optimal proxy variance. Then, each component satisfies $X_{i}=\boldsymbol{e}_{i}^{\top}\boldsymbol{X}\sim\text{Beta}(\alpha,(d-1)\alpha)$ and Theorem 1 shows that $\sigma_{i}^{2}$ is the optimal proxy variance for $X_{i}$ if and only if $\alpha=(d-1)\alpha$, i.e. if and only if $d=2$.

Acknowledgments

We wish to thank Stéphane Boucheron for an enlightening discussion of our results and an anonymous referee for insightful suggestions. O.M. would like to thank Université de Lyon, Université Jean Monnet and Institut Camille Jordan for financial support. This work was supported by the LABEX MILYON (ANR-10-LABX-0070) of Université de Lyon, within the program “Investissements d’Avenir” (ANR-11-IDEX-0007) operated by the French National Research Agency (ANR). J.A. would like to thank Inria Grenoble Rhône-Alpes and Laboratoire Jean Kuntzmann, Université Grenoble Alpes for financial support. J.A. is also member of Laboratoire de Statistique, CREST, Paris. This work was partially conducted during a scholar visit of J.A. at the Department of Statistics & Data Science of the University of Texas at Austin.