Large deviations for infinite weighted sums of stretched exponential random variables

TL;DR

This paper investigates the probabilities of rare large deviations in infinite weighted sums of independent random variables with stretched exponential tails, extending previous work to more general tail behaviors.

Contribution

It generalizes existing large deviation results to infinite sums with stretched exponential tails, beyond finite exponential moments.

Findings

Derived large deviation probabilities for infinite weighted sums with stretched exponential tails.

Extended previous finite sum results to infinite sums in this tail regime.

Provided new theoretical bounds for rare event probabilities in this setting.

Abstract

We study the large deviation probabilities of infinite weighted sums of independent random variables that have stretched exponential tails. This generalizes Kiesel and Stadtm\"uller (2000), who study the same objects under the assumption of finite exponential moments, and Gantert et al.\ (2014), who study finite weighted sums with stretched exponential tails.

Equations

$$\underset{n\to \infty }{\lim }\frac{1}{n}\log \mathbb{P}(\frac{1}{n}\sum _{i=1}^n{X}_i>x)=-\underset{t\in \mathbb{R}}{\sup }(tx-\log \varphi (t)),x>0.$$

$$\underset{n\to \infty }{\lim }\frac{1}{n^r}\log \mathbb{P}(\frac{1}{n}\sum _{i=1}^n{X}_i>x)=-\kappa (x-\mathbb{E}[X_1]{)}^r,x>\mathbb{E}[X_1].$$

$$\log \mathbb{P}(X>t)\sim -\kappa t^r,\text{as t→∞,}$$

$$\mathbb{P}\left(\sum _{i=1}^{\infty }{a}_i(n)X_i>x\right),\text{where x>0 and n→∞.}$$

$$\underset{n\to \infty }{\mathrm{limsup}}\mathbb{E}\left[\sum _{i=1}^{\infty }{a}_i(n)X_i\right]<x,$$

$$\underset{n\to \infty }{\lim }\sum _{i=1}^{\infty }{a}_i(n)=D\in [0,\infty )$$

$$\underset{n\to \infty }{\lim }{a}_{\max }(n{)}^r\log \mathbb{P}\left(\sum _{i=1}^{\infty }{a}_i(n)X_i>x\right)=-\kappa (x-D\cdot \mathbb{E}[X]{)}^r.$$

$$a_i(n):={\sigma }_i{\varphi }_n^{-1}1{\mathrm{l}}_{m_n\le i\le m_n+{\varphi }_n-1},$$

$$\mathbb{P}(\underset{i\ge 1}{\sup }{a}_i(n)X_i>x),n\to \infty ,$$

$$\underset{n\to \infty }{\mathrm{liminf}}{a}_{\max }(n{)}^r\log \mathbb{P}(\underset{i\ge 1}{\sup }{a}_i(n)X_i>x)\ge -\kappa x^r.$$

$$\mathbb{P}(\underset{i\ge 1}{\sup }{a}_i(n)X_i>x)\ge \mathbb{P}(a_{\max }(n)X_{m(n)}>x)=\mathbb{P}(X>x/a_{\max }(n)).$$

$$\underset{n\to \infty }{\lim }{a}_{\max }(n{)}^r\log \mathbb{P}(\underset{i\ge 1}{\sup }{a}_i(n)X_i>x)=-\kappa x^r.$$

$$\mathbb{P}(\underset{i\ge 1}{\sup }{a}_i(n)X_i>1)=\mathbb{P}(\bigcup _{i=1}^{\infty }\{a_i(n)X_i>1\})\le \sum _{i=1}^{\infty }\mathbb{P}(a_i(n)X_i>1).$$

$$\sum _{i=1}^{\infty }\mathbb{P}(a_i(n)X_i>1)=\sum _{i=1}^{\infty }\mathbb{P}(X>1/a_i(n))\le \sum _{i=1}^{\infty }C{e}^{-(\kappa -\epsilon )a_i(n{)}^{-r}},$$

$$\sum _{i=1}^{\infty }{e}^{-(\kappa -\epsilon )a_i(n{)}^{-r}}$$

$$\log \mathbb{P}(\underset{i\ge 1}{\sup }{a}_i(n)X_i>1)\le -(1-\epsilon )(\kappa -\epsilon )a_{\max }(n{)}^{-r}+\log [C(\epsilon (\kappa -\epsilon ){)}^{-2/r}(D+\epsilon )].$$

$$\mathbb{P}(\sum _{i=1}^{\infty }{a}_i(n)X_i>x)\ge \mathbb{P}(a_{\max }(n)X_{m(n)}>x)=\mathbb{P}(X>x/a_{\max }(n)).$$

$$\begin{gathered} \mathbb{P}(\sum _{i=1}^{\infty }{a}_i(n)X_i>x)\ge \mathbb{P}(a_{\max }(n)X_{m(n)}>x-\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)\mathbb{E}[X](1-\epsilon )) \\ \cdot \mathbb{P}(\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)X_i>\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)\mathbb{E}[X](1-\epsilon )). \end{gathered}$$

$$\underset{n\to \infty }{\mathrm{liminf}}{a}_{\max }(n{)}^r\log \mathbb{P}(a_{\max }(n)X_{m(n)}>x-\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)\mathbb{E}[X](1-\epsilon ))$$

$$\mathbb{P}(\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)X_i>\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)\mathbb{E}[X](1-\epsilon ))$$

$$\mathbb{P}(\sum _{i=1,i\ne m(n)}^{\infty }{a}_i(n)(X_i-\mathbb{E}[X_i])\le -\epsilon \mathbb{E}[X](D-\epsilon ))$$

$$\mathbb{P}(\sum _{i=1}^{\infty }{a}_i(n)X_i>1)$$

$$\mathbb{P}(\sum _{i=1}^{\infty }{a}_i(n)X_i>1,\underset{i\ge 1}{\sup }{a}_i(n)X_i\le A)$$

$$\sum _{i=1}^{\infty }\mathbb{E}[(e^{\lambda a_i(n)X}-1)1{\mathrm{l}}_{a_i(n)X\le A}]$$

$$\lambda :=\frac{B{A}^{r-1}}{a_{\max }(n{)}^r}.$$

$$\lambda a_i(n)X=\frac{B{A}^{r-1}{a}_i(n)}{a_{\max }(n{)}^r}\cdot X^{1-r}\cdot X^r\le \frac{B{A}^{r-1}{a}_i(n)}{a_{\max }(n{)}^r}\cdot \frac{A^{1-r}}{a_i(n{)}^{1-r}}\cdot X^r=\frac{B{a}_i(n{)}^r}{a_{\max }(n{)}^r}\cdot X^r.$$

$$\mathbb{E}[(e^{\lambda a_i(n)X}-1)1{\mathrm{l}}_{a_i(n)X\le A,\lambda a_i(n)X\ge \epsilon }]\le \mathbb{E}[(e^{\frac{B{a}_i(n{)}^r}{a_{\max }(n{)}^r}\cdot X^r}-1)\cdot 1{\mathrm{l}}_{\lambda a_i(n)X\ge \epsilon }].$$

$$\mathbb{E}[(e^{b{X}^r}-1)1{\mathrm{l}}_{X>a}]\le \frac{k}{1-b/B^{\prime }}{e}^{-(B^{\prime }-b)a^r},$$

$$\frac{k}{1-\frac{(\kappa -2\epsilon )a_i(n{)}^r}{(\kappa -\epsilon )a_{\max }(n{)}^r}}\exp (-(\kappa -\epsilon -(\kappa -2\epsilon )a_i(n{)}^r/a_{\max }(n{)}^r)\cdot [\epsilon a_i(n{)}^{-1}{\lambda }^{-1}{]}^r)$$

$$c_{\epsilon }\sum _{i=1}^{\infty }{e}^{-2K[a_i(n{)}^{-1}{a}_{\max }(n{)}^r{]}^r},$$

Full text

Keywords: independent, identically distributed random variables; large deviations; stretched exponential random variables; weighted sums

2010 Mathematics Subject Classification: 60F10

1 Introduction

A classical result in probability theory is Cramér’s theorem for the large deviations of sums of independent, identically distributed random variables: If $(X_{i})$ is an i.i.d. sequence with zero mean and for some $t>0$ the moment generating function $\phi(t):=\mathbb{E}\,e^{tX_{1}}$ is finite then

[TABLE]

It is also classical that Cramér’s theorem can be extended to a full large deviation principle; and it can be seen as the starting point of large deviation theory, see e.g. [6, 7].

Whenever the random variables $(X_{i})$ do not have any finite exponential moment, the behaviour of the large deviations is different. This is due to the fact that then the large deviation event is produced by only one variable being unusually large. The classical result here (cf. [14]) is as follows: if $(X_{i})$ is an i.i.d. sequence with stretched exponential tail, $\log{\mathbb{P}}(X_{1}>t)\sim-\kappa t^{r}$, as $t\to\infty$ for some $0<r<1$, and finite expectation then

[TABLE]

In this paper, we study weighted sums of i.i.d. random variables. There is quite some literature on large devations of weighted sums and their applications. The most recent general reference is Kiesel and Stadtmüller [12] (also see [1, 2, 3, 4, 5, 9, 10, 15] for further references). However, these papers deal with random variables that do have some finite exponential moment.

The only source, to the knowledge of the author, that deals with weighted sums of random variables that do not have any finite exponential moment is Gantert et al. [8]. There, finite sums of the type $\sum_{i=1}^{n}a_{i}(n)X_{i}$ are considered when the random variables have stretched exponential tails.

In this note, we treat the case of infinite weighted sums $\sum_{i=1}^{\infty}a_{i}(n)X_{i}$ with $(X_{i})$ i.i.d. random variables having stretched exponential tails. Besides filling this gap in the literature, the motivation comes from Baysian statistics: There, one is interested in proving contraction rates for the posterior distribution for nonparametric inverse problems. There, estimates of the type studied here are important, see e.g. Lemma 5.2 in [13], [16], or [11] for results with Gaussian priors, which require large deviation estimates of squared Gaussians, i.e. with exponential moments. We mention that the present results are directly motivated by a forthcoming work of S. Agapiou and P. Mathé in that area for non-Gaussian priors.

The paper is structured as follows. In Section 2, we define the concrete setup for this paper and state our main result. The proofs are given in Section 3.

2 Main result

Let $(a_{i}(n))_{i\geq 1,n=1,2,\ldots}$ be an array of non-negative numbers (let $\sup_{i}a_{i}(n)>0$ for all $n$ to avoid trivialities). Let $(X_{i})$ be a sequence of non-negative i.i.d. random variables, copies of the random variable $X$ with tail behaviour

[TABLE]

for some $0<r<1$ and $\kappa>0$.

We are interested in the probability

[TABLE]

The large deviation regime is characterized by the condition that the typical values of $\sum_{i=1}^{\infty}a_{i}(n)X_{i}$ lie below $x$, i.e.

[TABLE]

which we shall encode using assumption (4) below.

We can now formulate our main result, which is a “largest jump principle” for the large deviations of weighted sums of stretched exponential random variables. This means that the large deviation event is triggered by one of the terms in the sum being large, namely the one corresponding to the largest weight.

Theorem 2.1

Let $(X_{i})$ be a sequence of non-negative i.i.d. random variables, copies of $X$ with tail behaviour (2). Further, let $(a_{i}(n))_{i\geq 1,n=1,2,\ldots}$ be non-negative numbers with

[TABLE]

and such that $a_{\max}(n):=\max_{i\geq 1}a_{i}(n)>0$ and $a_{\max}(n)\to 0$. Then for any $x>D\cdot\mathbb{E}\,[X]$

[TABLE]

We stress that we do not need any regularity assumption on the sequence $(a_{i}(n))$. Note that $\max_{i\geq 1}a_{i}(n)$ exists (for any $n$), because (4) implies that $\lim_{i\to\infty}a_{i}(n)\to 0$.

Example 2.2

The classical result (1) is retrieved for $a_{i}(n)=n^{-1}1\mathrm{l}_{i\leq n}$.

Example 2.3

In a motivating example from Baysian statistics, $a_{i}(n)=1\mathrm{l}_{i\geq n}\sigma_{i}/\rho_{n}$, which gives the large deviation probability of “remainder” sums: ${\mathbb{P}}(\sum_{i=n}^{\infty}\sigma_{i}X_{i}>x\rho_{n})$. Here $(\sigma_{i})$ is a positive, summable sequence and $(\rho_{n})$ is a positive sequence.

Example 2.4

The work of Gantert et al. [8] in the case of non-negative random variables can be recovered as follows. They consider arrays with $a_{i}(n)=0$ for $i>n$. Their condition (B) implies that (4) holds with $D>0$ and that $n\cdot a_{\max}(n)\to s>0$. Note that we do not require $a_{\max}(n)$ to be of order $n^{-1}$ in this work.

Example 2.5

Examples where $a_{i}(n)$ depends on $n$ in a different way are given for instance by moving averages, where

[TABLE]

for positive sequences $\sigma$, $\phi$, $m$. Such objects were studied by [12] under the assumption of finite exponential moments (cf. the remark on p. 938 in [12]).

Possible extensions of the present results include the case that $X$ has a polynomial tail (rather than stretched exponential) or the precise behaviour for the case of a supremum rather than a sum (see Lemma 3.2 below for a partial result). In the spirit of Example 2.3, one could also consider $\sum_{i=N}^{\infty}$, where $N$ is random (cf. e.g. [2] for the case of finite sums). Further, one might want to add a slowly varying factor in (2).

3 Proofs

3.1 Auxiliary results for maxima

We start with two results for the rate of the probability

[TABLE]

which is the obvious analog of (3). We start with a lower bound.

Lemma 3.1

If $a_{\max}(n)\to 0$ then for any $x>0$

[TABLE]

If $\limsup_{n\to\infty}a_{\max}(n)>0$ then $\liminf_{n\to\infty}{\mathbb{P}}(\sup_{i\geq 1}a_{i}(n)X_{i}>x)>0$.

Proof: The claims follow immediately from the trivial estimate

[TABLE]

where $m(n):=\min\{i\geq 1:a_{i}(n)=a_{\max}(n)\}$. $\square$

We now turn to the corresponding upper bound. We shall prove it under more restrictive assumptions in order to avoid lengthy discussions (note that (4) is not necessary for the sup-problem). The stated lemma will be one ingredient in the proof of the main result.

Lemma 3.2

Assume that (4) holds and that $a_{\max}(n)\to 0$. Then we have for any $x>0$

[TABLE]

Proof: First note that we can assume w.l.o.g. that $x=1$, as otherwise it can be absorbed as a constant factor into the sequence $(a_{i}(n))$. The lower bound already follows from Lemma 3.1. For the upper bound, observe that

[TABLE]

Fix $0<\varepsilon<\kappa$. It remains to use the tail bound for $X$, which shows that the last term bounded from above as follows: For large enough $n$,

[TABLE]

with some constant $C>0$. The remainder of the proof consists in a treatment of this sum:

[TABLE]

where we used in the second step that $e^{-x}\leq x^{-2/r}$ for large $x$ and, in the last step, the assumptions that $\sum_{i=1}^{\infty}a_{i}(n)\to D$ and $a_{\max}(n)\to 0$. Combining this with (6) and (7) shows

[TABLE]

Multiplying by $a_{\max}(n)^{r}$, taking first $n\to\infty$ and then $\varepsilon\to 0$ shows the upper bound in the statement. $\square$

3.2 Proof of the main result

Here, we give the proofs of the lower and upper bound in Theorem 2.1, respectively.

Proof of the lower bound: Throughout, we use the notation $m(n):=\min\{i\geq 1:a_{i}(n)=a_{\max}(n)\}$.

Let us first treat the case that $D=0$. Then the lower bound already follows from assumption (2) together with

[TABLE]

Assume $D>0$. Then we can fix an $\varepsilon>0$ with $\varepsilon<D$. We begin by noting that

[TABLE]

Since $a_{\max}(n)\to 0$, (4) implies $\sum_{i=1,i\neq m(n)}^{\infty}a_{i}(n)=\sum_{i=1}^{\infty}a_{i}(n)-a_{\max}(n)\to D$. Therefore, the first term on the right-hand side of (8), by (2), satisfies

[TABLE]

We will show that the second term on the right-hand side of (8) tends to one for fixed $\varepsilon$ and $n\to\infty$. Combining this with the last formula will finish the proof of the lower bound in the theorem.

Note that, for large enough $n$,

[TABLE]

The last term tends to one, since by Chebyshev’s inequality

[TABLE]

which tends to zero (because the sum is bounded, by (4), and $a_{\max}(n)\to 0$), as required. $\square$

Proof of the upper bound: The first observation is that we can assume w.l.o.g. that $x=1$, as $x$ can be absorbed as a constant factor into the sequence $(a_{i}(n))$.

Step 1: Reduction step, main argument, overview.

Set $A:=x-D\mathbb{E}\,[X]=1-D\mathbb{E}\,[X]$ and note that $A>0$, by assumption. Further, fix $0<\varepsilon<\kappa/2$ such that also $1-(1+\varepsilon)D\mathbb{E}\,[X]>0$. First note that

[TABLE]

and the second term can be treated with Lemma 3.2, which shows that the second term has asymptotic order $\exp(-\kappa a_{\max}(n)^{-r}A^{r}(1+o(1)))$, as required by the assertion. If we can show that the first term is of the same or lower order, we obtain the statement.

Step 2: Exponential Chebychev inequality for the truncated random variables.

Let us consider the first term: For any $\lambda>0$, by the Markov inequality,

[TABLE]

Let us deal with the sum. Note that for $0\leq x\leq\varepsilon$ we have $e^{x}-1\leq\frac{e^{\varepsilon}-1}{\varepsilon}x\leq(1+\varepsilon)x$ (for $\varepsilon$ small enough). Thus

[TABLE]

Setting $B:=\kappa-2\varepsilon$ we shall use the last estimate with

[TABLE]

Step 3: We show that the second sum in (10) tends to zero for fixed $\varepsilon>0$ and $n\to\infty$.

First note that if $a_{i}(n)X\leq A$ then – using $r<1$ – we have

[TABLE]

Therefore,

[TABLE]

Further, it is elementary to show (see Lemma 3.3 below) that due to the tail estimate (2), which we use in the form ${\mathbb{P}}(X>t)\leq k\exp(-B^{\prime}t^{r})$ for all $t>0$ and some $k>0$, where $B^{\prime}:=\kappa-\varepsilon$, we have

[TABLE]

for any $a,b>0$ with $b<B^{\prime}$.

In our case, $b:=Ba_{i}(n)^{r}/a_{\max}(n)^{r}\leq B<B^{\prime}$ and $a:=\varepsilon a_{i}(n)^{-1}\lambda^{-1}$. Therefore, we see that the term on the right-hand side of (11) is bounded from above by

[TABLE]

The second sum in (10) is therefore bounded from above by

[TABLE]

where $2K=2K(\varepsilon):=\varepsilon^{1+r}B^{-r}A^{r(1-r)}$ and $c_{\varepsilon}:=k(\kappa-\varepsilon)/\varepsilon$. This can be treated as follows: Since $e^{-x}\leq x^{-1/r}$ for large enough $x$, we have

[TABLE]

Now, $\sum_{i=1}^{\infty}a_{i}(n)$ is bounded by assumption (4). Further, since $a_{\max}(n)\to 0$, the term $e^{-Ka_{\max}(n)^{-(1-r)r}}a_{\max}(n)^{-r}$ tends to zero for fixed $\varepsilon$ and $n\to\infty$. This finishes the proof of the fact that the second sum in (10) tends to zero.

Step 4: Final computations. Putting Step 3 together with (9) and (10), we have seen that for fixed $\varepsilon$ and $n\to\infty$

[TABLE]

Multiplying by $a_{\max}(n)^{r}$ and using (4), we obtain

[TABLE]

Letting $\varepsilon\to 0$ shows the assertion. $\square$

During the course of the last proof, we used the following completely elementary lemma.

Lemma 3.3

Let $X$ be a non-negative random variable with ${\mathbb{P}}(X>t)\leq ke^{-B^{\prime}t^{r}}$ for all $t>0$ and $k,B^{\prime},r>0$. Then, for any $a>0$ and any $0<b<B^{\prime}$,

[TABLE]

Proof: Note that

[TABLE]

$\square$

Acknowledgement. The author is indebted to Sergios Agapiou (University of Cyprus) and Peter Mathé (WIAS Berlin) for bringing this problem to his attention and to Marvin Kettner (Darmstadt) for valuable suggestions.