A variance-sensitive Gaussian concentration inequality

TL;DR

This paper introduces a Gaussian concentration inequality tailored for non-Lipschitz functions, extending existing results in one dimension to broader classes of functions.

Contribution

It provides a new concentration inequality for non-Lipschitz functions, complementing prior work by Paouris and Valettas in one-dimensional settings.

Findings

Extended Gaussian concentration bounds for non-Lipschitz functions.

Complemented existing inequalities in one-dimensional cases.

Broadens applicability of concentration inequalities.

Abstract

In this note, we obtain a Gaussian concentration inequality for a class of non-Lipschitz functions. In the one-dimensional case, our results supplement those established by Paouris and Valettas in [8].

Equations

$$P\left(f(Z)-Ef(Z)\le -t\right)\le \exp \left(-\frac{t^2}{2L^2}\right)$$

$$P\left(f(Z)-M\le -t\right)\le \frac{1}{2}\exp \left(-\frac{t^2}{2L^2}\right),$$

$$P\left(f(Z)-M\le -tE(f(Z)-M{)}_{+}\right)\le \Phi \left(-\frac{\sqrt{2\pi }}{32}t\right)\forall t>0.$$

$$P\left(f(Z)-M\le -t\right)\le \frac{1}{2}\exp \left(-\frac{\pi t^2}{1024\mathrm{Var}f(Z)}\right)\forall t>0$$

$$P\left(f(Z)-Ef(Z)\le -t\right)$$

$$<\exp \left(-\frac{t^2}{1000\mathrm{Var}f(Z)}\right)\forall t>\sqrt{\mathrm{Var}f(Z)}.$$

$$T_{ij}(z):={\int }_0^{1}E\left[\sum _{k=1}^n\frac{\partial f_i}{\partial z_k}(z)\frac{\partial f_j}{\partial z_k}(\alpha z+\sqrt{1-{\alpha }^2}{Z}^{\prime })\right]d\alpha ,z\in {\mathbb{R}}^n,$$

$$T_{lij}(z):={\int }_0^{1}E\left[\sum _{k=1}^n\frac{\partial f_l}{\partial z_k}(z)\frac{\partial T_{ij}}{\partial z_k}(\alpha z+\sqrt{1-{\alpha }^2}{Z}^{\prime })\right]d\alpha ,z\in {\mathbb{R}}^n.$$

$$P\left(f_1(Z)-E{f}_1(Z)\le -t_1,\cdots ,f_d(Z)-E{f}_d(Z)\le -t_d\right)\le \exp \left(-\frac{t_1^2+\cdots +t_d^2}{2\Vert \Sigma {\Vert }_{\mathrm{op}}}\right),$$

$$P\left(f_1(Z)-E{f}_1(Z)\le -t_1,\cdots ,f_d(Z)-E{f}_d(Z)\le -t_d\right)\le \exp \left(-\frac{1}{2}t{\Sigma }^{-1}{t}^{⊺}\right),$$

$$P$$

$$\le P\left(f_1(Z)-E{f}_1(Z)\le -t_1+M_1-E{f}_1(Z),\cdots ,f_d(Z)-E{f}_d(Z)\le -t_d+M_d-E{f}_d(Z)\right)$$

$$\le P\left(f_1(Z)-E{f}_1(Z)\le -t_1+\sqrt{\mathrm{Var}{f}_1(Z)},\cdots ,f_d(Z)-E{f}_d(Z)\le -t_d+\sqrt{\mathrm{Var}{f}_d(Z)}\right)$$

$$\le \exp \left(-\frac{(t_1-\sqrt{\mathrm{Var}{f}_1(Z)}{)}^2+\cdots +(t_d-\sqrt{\mathrm{Var}{f}_d(Z)}{)}^2}{2\Vert \Sigma {\Vert }_{\mathrm{op}}}\right)$$

$$P$$

$$\le P\left(f_1(Z)-E{f}_1(Z)\le -t_1,\cdots ,f_d(Z)-E{f}_d(Z)\le -t_d\right)$$

$$\le \exp \left(-\frac{t_1^2+\cdots +t_d^2}{2\Vert \Sigma {\Vert }_{\mathrm{op}}}\right)\forall (t_1,\cdots ,t_d)\in {\mathbb{R}}_{+}^d.$$

$$P\left(f_1(Z)-E{f}_1(Z)\le -t\right)\le \exp \left(-\frac{t^2}{2\mathrm{Var}{f}_1(Z)}\right)\forall t\ge 0,$$

$$P\left(\underset{1\le i\le d}{\min }(h_i(Z)-E{h}_i(Z))>t\right)$$

$$\le \exp \left(-\frac{1}{2}{t}^2({\theta }_1+\cdots +{\theta }_d)\right)\forall t\ge 0,$$

$$\frac{\partial T_{ij}}{\partial z_k}(z)={\int }_0^{1}E\left[\sum _{u=1}^n\frac{{\partial }^2{f}_i}{\partial z_k\partial z_u}(z)\frac{\partial f_j}{\partial z_u}(\alpha z+\sqrt{1-{\alpha }^2}{Z}^{\prime })+\sum _{u=1}^n\frac{\partial f_i}{\partial z_u}(z)\frac{{\partial }^2{f}_j}{\partial z_k\partial z_u}(\alpha z+\sqrt{1-{\alpha }^2}{Z}^{\prime })\right]d\alpha .$$

$$\frac{{\partial }^2{f}_i}{\partial z_k\partial z_u}\ge 0\forall k,u=1,\cdots ,n.$$

$$\Sigma=\left(\begin{array}[]{cc}2&0\\ 0&2\\ \end{array}\right).$$

$$P\left(\min \{Z_1+Z_2,Z_1-Z_2\}>t\right)\le \exp \left(-\frac{t^2}{2}\right)\forall t\ge 0.$$

$$P\left(\min \{Z_1+Z_2,Z_1-Z_2\}>t\right)$$

$$=(1-\Phi (t/\sqrt{2}){)}^2\forall t\ge 0,$$

$$\mathcal{P}:=\bigg{\{}p(x)=\sum\limits_{i=1}^{N}a_{i}x^{2m_{i}},x\in\mathbb{R}:a_{i}\in\mathbb{R}_{+}\,\,\,\text{and}\,\,\,m_{i}\in\mathbb{N}_{+},i=1,\cdots,N\bigg{\}}$$

$$\mathcal{Q}:=\bigg{\{}\sum\limits_{i=1}^{n}p_{i}(Z_{i}):p_{i}\in\mathcal{P}\bigg{\}}.$$

$$z_k\mapsto {\int }_0^{1}E\left[\sum _{k=1}^n{z}_k^{2m_1-1}(\alpha z_k+\sqrt{1-{\alpha }^2}{Z}_k^{\prime }{)}^{2m_2-1}\right]d\alpha$$

$$u(z):={\int }_0^{1}E\left[\sum _{k=1}^n\frac{\partial f}{\partial z_k}(z)\frac{\partial g}{\partial z_k}(\alpha z+\sqrt{1-{\alpha }^2}{Z}^{\prime })\right]d\alpha ,z\in {\mathbb{R}}^n.$$

$$\mathrm{Cov}(f(Z),g(Z))=E[u(Z)].$$

$$E\left[\exp \left(-\sum _{i=1}^d{\theta }_i{f}_i(Z)\right)\right]\le E\left[\exp \left(\sum _{i=1}^d{\theta }_i{N}_i\right)\right],$$

$$\phi (t):=E\left[\exp \left(-\sqrt{t}\sum _{i=1}^d{\theta }_i{f}_i(Z)+\sqrt{1-t}\sum _{i=1}^d{\theta }_i{N}_i\right)\right],t\in [0,1].$$

$${\phi }^{\prime }(t)=E\left[\exp \left(-\sqrt{t}\sum _{i=1}^d{\theta }_i{f}_i(Z)+\sqrt{1-t}\sum _{i=1}^d{\theta }_i{N}_i\right)\left(-\frac{1}{2\sqrt{t}}\sum _{i=1}^d{\theta }_i{f}_i(Z)-\frac{1}{2\sqrt{1-t}}\sum _{i=1}^d{\theta }_i{N}_i\right)\right].$$

Full text

Keywords: Concentration of measure, Gaussian vectors.

2010 Mathematics Subject Classification: Primary 60E15.

1 Introduction

In the whole paper, let $Z=(Z_{1},\cdots,Z_{n})$ be an $n$-dimensional standard Gaussian vector and $Z^{\prime}=(Z^{\prime}_{1},\cdots,Z^{\prime}_{n})$ be an independent copy of $Z.$ It is well known that if $f:\mathbb{R}^{n}\to\mathbb{R}$ is a Lipschitz function with Lipschitz constant $L,$ then we have the following concentration inequalities (see e.g. [3, 10]), for all $t>0,$

[TABLE]

and

[TABLE]

where $M$ is a median for $f(Z).$ In a recent paper [8], Paouris and Valettas pointed out that Gaussian concentration phenomenon also holds for the convex functions. More specifically, they got the following.

Theorem A. Let $f:\mathbb{R}^{n}\to\mathbb{R}$ be a convex function with $E|f(Z)|<\infty$ and let $M$ be a median for $f(Z).$ Then, it holds that

[TABLE]

As discussed in [8], for any convex function $f$ with $E|f(Z)|^{2}<\infty,$ the inequality (1.3) implies the following estimates

[TABLE]

and

[TABLE]

Interestingly, the bound (1.4) and (1.5) depend on ${\rm Var}f(Z)$ instead of Lipschitz constant $L.$ Hence, those bound improve (1.1) and (1.2). Here the improvement lies in the fact that ${\rm Var}f(Z)\leq L^{2}$ (by the Gaussian Poincaré inequality) and one can find many examples of $f$ for which ${\rm Var}f(Z)\ll L^{2}.$ For further results, Valettas proved in [14] that (1.5) is tight if the convex function $f$ is not superconcentrated, some variance-sensitive concentration inequalities for the log-concave probability measures were obtained in [9, 14]. We also refer to [1, 11] for other concentration results.

In the present paper, our purpose is to obtain a multi-dimensional version of the variance-sensitive concentration inequality (1.5). Our main result is the following statement.

Theorem 1.1**.**

Let $f_{i}:\mathbb{R}^{n}\to\mathbb{R},i=1,\cdots,d$ be twice differentiable functions. For each $i,j,l=1,\cdots,d,$ we define the functions

[TABLE]

and

[TABLE]

Assume that

(i) $E\left[\exp\left(-\sum\limits_{i=1}^{d}\theta_{i}f_{i}(Z)\right)\right]<\infty$ for any $\theta_{1},\cdots,\theta_{d}\geq 0,$ and all derivatives of $f_{i}^{\prime}s$ have subexponential growth at infinity,

(ii) $T_{lij}(z)\geq 0$ for every $i,j,l=1,\cdots,d,$ and $z\in\mathbb{R}^{n}.$

Then, for any $t=(t_{1},\cdots,t_{d})\in\mathbb{R}^{d}_{+},$ we have

[TABLE]

where $\Sigma$ is covariance matrix of $(f_{1}(Z),\cdots,f_{d}(Z))$ and $\|.\|_{\rm op}$ denotes the operator norm.

Assume, further, that the inverse matrix $\Sigma^{-1}$ exists, then the bound (1.6) can be improved to the following

[TABLE]

for any $t$ satisfying $t\,\Sigma^{-1}\in\mathbb{R}^{d}_{+},$ where $t^{\intercal}$ is the transpose of $t.$

The proof of Theorem 1.1 is deferred to Section 2. To see clearer our new contributions let us end up this Section with some remarks and examples.

Remark 1.1*.*

Let $M_{i}$ be a median for $f_{i}(Z),i=1,\cdots,d.$ Taking into account the fact that $M_{i}-Ef_{i}(Z)\leq E|f_{i}(Z)-M_{i}|\leq\sqrt{{\rm Var}f_{i}(Z)}$ we can get concentration inequalities about the median. Indeed, for example, we obtain from (1.6) that

[TABLE]

for all $t_{i}\geq\sqrt{{\rm Var}f_{i}(Z)},i=1,\cdots,d.$ Furthermore, by a result of Kwapień [7], if $f_{i}^{\prime}s$ are convex functions then $M_{i}-Ef_{i}(Z)\leq 0$ and hence,

[TABLE]

Remark 1.2*.*

In the one-dimensional case ($d=1$), the bound (1.6) reduces to

[TABLE]

provided that $T_{111}(z)\geq 0$ for all $z\in\mathbb{R}^{n}.$ In particular, (1.8) holds true if $\frac{\partial f_{1}(z)}{\partial z_{i}}\geq 0$ and $\frac{\partial^{2}f_{1}(z)}{\partial z_{i}\partial z_{j}}\geq 0$ for all $i,j=1,\cdots,n.$ We note that the class of functions $f_{1}$ satisfying $T_{111}(z)\geq 0$ is not the same as the class of convex functions, see the examples provided by Tanguy in [13, p. 981]. Hence, our bound (1.8) supplements the bound (1.5).

Remark 1.3*.*

The concentration inequality (1.8) was proved first in [6], which is a draft version of the present paper. We also refer the reader to [13, Sec. 5] for discussions and a neater proof using semigroup operators.

Remark 1.4*.*

In the multi-dimensional setting, an important application of Theorem 1.1 is to estimate the tail of the minimal component of non-Gaussian random vectors. Let $h_{i}:\mathbb{R}^{n}\to\mathbb{R},i=1,\cdots,d$ be twice differentiable functions. Assume that the covariance matrix $\Sigma$ of $(h_{1}(Z),\cdots,h_{d}(Z))$ is invertible and the functions $f_{i}=-h_{i},i=1,\cdots,d$ satisfy the conditions $(i)$-$(ii)$ of Theorem 1.1. Then, for $(\theta_{1},\cdots,\theta_{d}):=\Sigma^{-1}\,(1,\cdots,1)^{\intercal},$ the inequality (1.7) gives us

[TABLE]

provided that $\theta_{1}+\cdots+\theta_{d}>0.$ Note that (1.9) is similar to the relation (3.17) given in [4] where Chakrabarty & Samorodnitsky establish the precise asymptotic behaviour for the tail of the minimal component of the Gaussian vector.

Remark 1.5*.*

Observe that

[TABLE]

Hence, we can verify the condition $(ii)$ of Theorem 1.1 by checking the following

$(ii^{\prime})$ For all $i=1,\cdots,d,$ $f_{i}$ is coordinatewise non-decreasing (or non-increasing) and

[TABLE]

For example, the sum of log-normal random variables fulfills the condition $(ii^{\prime}).$

Remark 1.6*.*

Let $U=(U_{1},\cdots,U_{n})$ be a Gaussian random vector (coordinates are not necessarily independent). It is well known that we can express $U=AZ,$ where $A$ is a matrix such that $AA^{\intercal}$ is the covariance matrix of $U.$ Hence, Theorem 1.1 can be extended to the functions of arbitrary Gaussian random vectors.

Example 1.1**.**

Consider $h_{1}(Z):=Z_{1}+Z_{2}$ and $h_{2}(Z):=Z_{1}-Z_{2},$ we have

[TABLE]

Hence, the bound (1.9) becomes

[TABLE]

On the other hand, we observe that $Z_{1}+Z_{2}$ and $Z_{1}-Z_{2}$ are independent Gaussian random variables. We obtain

[TABLE]

where $\Phi$ is the cumulative distribution function of a standard normal random variable. Thus, in this simple example, our estimate (1.10) is sharp.

Example 1.2**.**

In this example, we construct a class of random vectors satisfying Theorem 1.1. Fixed $N\in\mathbb{N}_{+}=\{1,2,\cdots\},$ define the set $\mathcal{P}$ of the polynomials of the form

[TABLE]

and denote by $\mathcal{Q}$ the following set of non-negative random variables

[TABLE]

We observe that, for any $m_{1},m_{2}\in\mathbb{N}_{+},$ the function

[TABLE]

belongs to $\mathcal{P}$ (this is due to the fact that $E[(Z_{k}^{\prime})^{2m-1}]=0$ for any $m\in\mathbb{N}_{+}$). Hence, it is easy to see that if the random variables $f_{i}(Z),f_{j}(Z)\in\mathcal{Q},$ then $T_{ij}(Z)$ also belongs to $\mathcal{Q}.$ As a consequence, we can conclude that every vector $(f_{1}(Z),\cdots,f_{d}(Z))\in\mathcal{Q}^{d}$ will satisfy both conditions $(i)$ and $(ii)$ of Theorem 1.1.

2 Proofs

The key tool in our proof is the covariance identify formula for Gaussian functionals. We have

Lemma 2.1**.**

Let $f,g:\mathbb{R}^{n}\to\mathbb{R}$ be differentiable functions. Suppose that $f,g$ and their derivatives have subexponential growth at infinity. Define

[TABLE]

Then, we have

[TABLE]

Proof.

Note that the subexponential growth condition ensures the existence of all expectations. Hence, the desired result follows directly from Lemma 2.1.4 in [2] and the routine approximation argument. ∎

For the proof of Theorem 1.1, we need the following technical lemma.

Lemma 2.2**.**

Let $f_{i}:\mathbb{R}^{n}\to\mathbb{R},i=1,\cdots,d$ be twice differentiable functions satisfying the conditions $(i)$-$(ii)$ of Theorem 1.1. Then, for any $\theta_{1},\cdots,\theta_{d}\geq 0,$ we have

[TABLE]

where $N=(N_{1},\cdots,N_{d})$ is a Gaussian vector satisfying $E[N_{i}]=E[f_{i}(Z)]$ and $E[N_{i}N_{j}]=E[f_{i}(Z)f_{j}(Z)]$ for all $i,j=1,\cdots,d.$

Proof.

Without loss of generality, we can assume that $N$ is independent of $(f_{1}(Z),\cdots,f_{d}(Z))$ and $E[N_{i}]=E[f_{i}(Z)]=0$ for all $i=1,\cdots,d.$ Consider the function

[TABLE]

This function is well-defined because of the condition $(i)$ of Theorem 1.1. We have, for $t\in(0,1),$

[TABLE]

By the independence, we obtain

[TABLE]

An application of Lemma 2.1 yields

[TABLE]

where the function $T$ is defined by

[TABLE]

On the other hand, by the integration by parts formula for Gaussian random variables (see, e.g. Appendix A.6 in [12]) we obtain

[TABLE]

where $\sigma_{ij}:={\rm Cov}(N_{i},N_{j})={\rm Cov}(f_{i}(Z),f_{j}(Z)).$ By inserting (2.3) and (2.4) into (2.2) we get

[TABLE]

We now observe that ${\rm Cov}(f_{i}(Z),f_{j}(Z))=E[T_{ij}(Z)]$ for every $\,\,i,j=1,\cdots,d.$ Hence,

[TABLE]

So, once again, we can apply Lemma 2.1 and we obtain

[TABLE]

where $\bar{T}$ is defined by

[TABLE]

Consequently, the condition $(ii)$ of Theorem 1.1 implies that $\bar{T}(z)\geq 0,$ and hence, $\varphi^{\prime}(t)\leq 0,\,\,t\in(0,1).$ So the claim (2.1) holds true because

[TABLE]

This completes the proof of the lemma. ∎

Remark 2.1*.*

The proof of Lemma 2.2 has some similarities with the proof of Slepian’s lemma, see e.g. [5]. The main difference lies in the fact that we applied Lemma 2.1 twice times. This is the key idea allowing us to handle non-Lipschitz functions.

Remark 2.2*.*

The left hand side of (2.1) is the Laplace transformation of $(f_{1}(Z),\cdots,f_{d}(Z)),$ and hence, the estimate (2.1) could be useful for other research problems.

Proof of Theorem 1.1. Without loss of generality, we may and will assume that $Ef_{i}(Z)=0$ for every $i=1,\cdots,d.$ Then, by Markov’s inequality we have

[TABLE]

for all $t=(t_{1},\cdots,t_{d})\in\mathbb{R}^{d}_{+}.$ From (2.1) and (2.5) we obtain, for any $\theta=(\theta_{1},\cdots,\theta_{d})\in\mathbb{R}^{d}_{+},$

[TABLE]

Taking the infimum over all $\theta\in\mathbb{R}^{d}_{+},$ the optimal choice of $\theta$ is $t/\|\Sigma\|_{\rm op}.$ So (1.6) follows from (2.7). Similarly, if $\Sigma^{-1}$ exists, we obtain (1.7) from (2.6) by choosing $\theta=t\,\Sigma^{-1}.$

The proof of Theorem 1.1 is completed. $\square$