On Stein's lemma in hypotheses testing in general non-asymptotic case

TL;DR

This paper investigates the optimal exponential decay rate of type-II error in hypothesis testing within general probability spaces, providing non-asymptotic bounds and insights into Stein's exponent without relying on asymptotic assumptions.

Contribution

It derives non-asymptotic bounds for the decay rate of type-II error in hypothesis testing, extending Stein's lemma to general probability spaces without asymptotic limits.

Findings

Non-asymptotic bounds for error decay rates

Convergence rate analysis for Stein's exponent

Illustrative examples demonstrating bounds

Abstract

The problem of testing two simple hypotheses in a general probability space is considered. For a fixed type-I error probability, the best exponential decay rate of the type-II error probability is investigated. In regular asymptotic cases (i.e., when the length of the observation interval grows without limit) the best decay rate is given by Stein's exponent. In the paper, for a general probability space, some non-asymptotic lower and upper bounds for the best rate are derived. These bounds represent pure analytic relations without any limiting operations. In some natural cases, these bounds also give the convergence rate for Stein's exponent. Some illustrating examples are also provided.

Equations

$$\begin{array}{c}{\mathcal{H}}_0:\mathbf{x}\sim \mathbf{P},\\ {\mathcal{H}}_1:\mathbf{x}\sim \mathbf{Q}.\end{array}$$

$$\begin{array}{c}\mathbf{x}\in \mathcal{D}\Rightarrow {\mathcal{H}}_0,\mathbf{x}\notin \mathcal{D}\Rightarrow {\mathcal{H}}_1,\end{array}$$

$$\alpha (\mathcal{D})=\mathbf{P}(\mathbf{x}\notin \mathcal{D}|{\mathcal{H}}_0)$$

$$\begin{array}{c}\beta (\mathcal{D})=\mathbf{P}(\mathbf{x}\in \mathcal{D}|{\mathcal{H}}_1).\end{array}$$

$$\beta (\alpha )=\underset{\mathcal{D}:\alpha (\mathcal{D})\le \alpha }{\inf }\beta (\mathcal{D}).$$

$$\begin{array}{c}D(\mathbf{P}||\mathbf{Q})={\mathbf{E}}_{\mathbf{P}}\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\ge 0,\end{array}$$

$$\begin{array}{c}{\mathcal{H}}_0:x_i\sim \mathbf{P},i=1,\dots ,n;\\ {\mathcal{H}}_1:x_i\sim \mathbf{Q},i=1,\dots ,n.\end{array}$$

$$\begin{array}{c}\underset{\alpha \to 0}{\lim }\underset{n\to \infty }{\lim }\frac{1}{n}\ln \beta (\alpha )=-D(\mathbf{P}||\mathbf{Q}).\end{array}$$

$$\begin{array}{c}\ln \beta (\alpha )\ge -\frac{D(\mathbf{P}||\mathbf{Q})+h(\alpha )}{1-\alpha },\\ h(\alpha )=-\alpha \ln \alpha -(1-\alpha )\ln (1-\alpha ),\end{array}$$

$$\begin{array}{c}\ln \beta (\alpha )\le -D(\mathbf{P}||\mathbf{Q})+{\mu }_0(\alpha ),\end{array}$$

$$\begin{array}{c}{\mathbf{P}}_{\mathbf{P}}\left\{\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\le D(\mathbf{P}||\mathbf{Q})-{\mu }_0\right\}\le \alpha .\end{array}$$

$$\begin{array}{c}\ln \beta (\alpha )\le -D(\mathbf{P}||\mathbf{Q})+{\alpha }^{-1/2}{r}_1(\mathbf{P},\mathbf{Q}),\end{array}$$

$$\begin{array}{c}{r}_1(\mathbf{P},\mathbf{Q})={\left\{{\mathbf{E}}_{\mathbf{P}}{\left[\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\right]}^2-D^2(\mathbf{P}||\mathbf{Q})\right\}}^{1/2}.\end{array}$$

$$\begin{array}{c}\beta =\mathbf{Q}(\mathcal{D})=\underset{\mathcal{D}}{\int }\frac{d\mathbf{Q}}{d\mathbf{P}}(\mathbf{x})d\mathbf{P}(\mathbf{x}),\alpha =\mathbf{P}({\mathcal{D}}^c).\end{array}$$

$$\begin{array}{c}\ln \frac{\beta }{1-\alpha }=\ln \left[\frac{1}{(1-\alpha )}\underset{\mathcal{D}}{\int }\frac{d\mathbf{Q}}{d\mathbf{P}}(\mathbf{x})d\mathbf{P}(\mathbf{x})\right]\ge \\ \ge \frac{1}{(1-\alpha )}\underset{\mathcal{D}}{\int }\ln \frac{d\mathbf{Q}}{d\mathbf{P}}(\mathbf{x})d\mathbf{P}(\mathbf{x})=\\ =-\frac{D(\mathbf{P}||\mathbf{Q})}{1-\alpha }-\frac{1}{(1-\alpha )}\underset{{\mathcal{D}}^c}{\int }\ln \frac{d\mathbf{Q}}{d\mathbf{P}}(\mathbf{x})d\mathbf{P}(\mathbf{x}).\end{array}$$

$$\begin{array}{c}\frac{1}{(1-\alpha )}\underset{{\mathcal{D}}^c}{\int }\ln \frac{d\mathbf{Q}}{d\mathbf{P}}(\mathbf{x})d\mathbf{P}(\mathbf{x})\le \\ \le \frac{\alpha }{(1-\alpha )}\ln \left[\frac{1}{\alpha }\underset{{\mathcal{D}}^c}{\int }d\mathbf{Q}(\mathbf{x})\right]=\frac{\alpha }{(1-\alpha )}\ln \frac{1-\beta }{\alpha }\le \frac{\alpha }{(1-\alpha )}\ln \frac{1}{\alpha }.\end{array}$$

$$\begin{array}{c}\ln \frac{\beta }{1-\alpha }\ge -\frac{D(\mathbf{P}||\mathbf{Q})}{1-\alpha }-\frac{\alpha }{(1-\alpha )}\ln \frac{1}{\alpha },\end{array}$$

$$\begin{array}{c}{\mathcal{A}}_{\mu }=\left\{\mathbf{x}\in \mathcal{X}:\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\ge D(\mathbf{P}||\mathbf{Q})-\mu \right\}.\end{array}$$

$$\begin{array}{c}{\beta }_{\mu }=\underset{{\mathcal{A}}_{\mu }}{\int }d\mathbf{Q}=\underset{{\mathcal{A}}_{\mu }}{\int }\frac{d\mathbf{Q}}{d\mathbf{P}}(\mathbf{x})d\mathbf{P}\le e^{-D(\mathbf{P}||\mathbf{Q})+\mu }.\end{array}$$

$$\begin{array}{c}{\alpha }_{\mu }={\mathbf{P}}_{\mathbf{P}}\left\{\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\le D(\mathbf{P}||\mathbf{Q})-\mu \right\}=\mathbf{P}(\eta \ge \mu ),\end{array}$$

$$\eta =D(\mathbf{P}||\mathbf{Q})-\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x}),\mathbf{E}\eta =0.$$

$$\begin{array}{c}{\alpha }_{{\mu }_0(\alpha )}=\alpha ,\end{array}$$

$$\begin{array}{c}\ln {\beta }_{{\mu }_0(\alpha )}\le -D(\mathbf{P}||\mathbf{Q})+{\mu }_0(\alpha ),\end{array}$$

$$\begin{array}{c}\mathbf{P}(\eta \ge \mu )\le {\mu }^{-2}\mathbf{E}({\eta }_{+}^2),\end{array}$$

$$\begin{array}{c}{\alpha }_{\mu }\le {\mu }^{-2}{\mathbf{E}}_{\mathbf{P}}{\left[D(\mathbf{P}||\mathbf{Q})-\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\right]}_{+}^2.\end{array}$$

$$\begin{array}{c}{\mu }_1={\alpha }^{-1/2}{\left\{{\mathbf{E}}_{\mathbf{P}}{\left[D(\mathbf{P}||\mathbf{Q})-\ln \frac{d\mathbf{P}}{d\mathbf{Q}}(\mathbf{x})\right]}_{+}^2\right\}}^{1/2}=\\ ={\alpha }^{-1/2}{r}_1(\mathbf{P},\mathbf{Q}).\end{array}$$

$$\begin{array}{c}{\alpha }_{{\mu }_1}\le \alpha ,\end{array}$$

$$\begin{array}{c}\ln {\beta }_{{\mu }_1}\le -D(\mathbf{P}||\mathbf{Q})+{\mu }_1=\\ =-D(\mathbf{P}||\mathbf{Q})+{\alpha }^{-1/2}{r}_1(\mathbf{P},\mathbf{Q}),\end{array}$$

$$\begin{array}{rl}& {\mathcal{H}}_0:{\mathbf{x}}_T=x_{{\mathbf{p}}_T},\\ & {\mathcal{H}}_1:{\mathbf{x}}_T=x_{{\mathbf{q}}_T},\end{array}$$

$$\begin{array}{c}\frac{d{\mathbf{P}}_{{\mathbf{p}}_T}}{d{\mathbf{P}}_{{\mathbf{q}}_T}}(\mathbf{x})=\exp \left\{(\ln ({\mathbf{p}}_T/{\mathbf{q}}_T),dx)-({\mathbf{p}}_T-{\mathbf{q}}_T,1)\right\},\end{array}$$

Full text

Statistical Inference for Stochastic Processes,

25, no. , pp. , 2022,   DOI: 10.1007/s11203-022-09278-4.

M. V. BURNASHEV 111Burnashev M. V. is with Institute for Information Transmission Problems, Russian Academy of Sciences, Moscow; email: burn@iitp.ru

ON STEIN’S LEMMA IN HYPOTHESES TESTING IN GENERAL NON-ASYMPTOTIC CASE 222This work was supported by the Russian Foundation for Basic Research under Grant 19-01-00364.

**Abstract – ** The problem of testing two simple hypotheses in a general probability space is considered. For a fixed type-I error probability, the best exponential decay rate of the type-II error probability is investigated. In regular asymptotic cases (i.e., when the length of the observation interval grows without limit) the best decay rate is given by Stein’s exponent. In the paper, for a general probability space, some non-asymptotic lower and upper bounds for the best rate are derived. These bounds represent pure analytic relations without any limiting operations. In some natural cases, these bounds also give the convergence rate for Stein’s exponent. Some illustrating examples are also provided.

**Index Terms – ** Testing of hypotheses, type-I and type-II error probabilities, Stein’s exponent.

1 Introduction and Main result

We assume that there are given probability measures $\mathbf{P}$ and $\mathbf{Q}$ on a measurable space $(\cal X,{\cal B})$. We consider the problem of testing the simple hypothesis $\mathcal{H}_{0}$ against the simple alternative $\mathcal{H}_{1}$, based on observation ${\mathbf{x}}$:

[TABLE]

If for the testing of hypotheses $\mathcal{H}_{0}$ and $\mathcal{H}_{1}$ a decision (acceptance) region ${\mathcal{D}}\in{\cal X}$ is chosen, such that

[TABLE]

then the type-I error probability $\alpha({\mathcal{D}})$ and the type-II error probability $\beta({\mathcal{D}})$ are defined by formulas, respectively,

[TABLE]

and

[TABLE]

In the paper, we consider the case when the type-I error probability $\alpha$, $0<\alpha<1$, is fixed, and we are interested in the minimal possible type-II error probability $\beta(\alpha)$

[TABLE]

The corresponding optimal decision region ${\mathcal{D}}(\alpha)$ to this problem is given by Neyman-Pearson lemma [1, 2].

We will need the following notion [3].

Definition 1. * For probability distributions ${\mathbf{P}}$ and ${\mathbf{Q}}$ on a measurable space $(\cal X,{\cal B})$ the function $D({\mathbf{P}}||{\mathbf{Q}})$ (Kullback–Leibler distance or divergence for measures ${\mathbf{P}}$ and ${\mathbf{Q}}$) is defined as*

[TABLE]

where the expectation is taken over the measure ${\mathbf{P}}$.

In order to describe Stein’s lemma and its relation to the value $\beta(\alpha)$ via the function $D({\mathbf{P}}||{\mathbf{Q}})$, we consider first the case ${\cal X}={\mathbf{R}}^{n}$, when an observation ${\mathbf{x}}$ from (1) has the form ${\mathbf{x}}={\mathbf{x}}_{n}=(x_{1},\ldots,x_{n})\in{\mathbf{R}}^{n}$. We assume that the sample ${\mathbf{x}}_{n}$ consists of independent and identically distributed (i.i.d.) random variables $x_{i}$, $i=1,\ldots,n$ with a given distribution ${\mathbf{P}}$ (in the case of $\mathcal{H}_{0}$), or with a given distribution ${\mathbf{Q}}$ (in the case of $\mathcal{H}_{1}$):

[TABLE]

If, in model (7) ${\mathbf{P}}\not\equiv{\mathbf{Q}}$, then $D({\mathbf{P}}||{\mathbf{Q}})>0$ and the value $\beta(\alpha)$ decreases exponentially as $n\to\infty$. Moreover, we have [3, Theorem 3.3]

[TABLE]

Relation (8) is called Stein’s lemma [4], [5]. 111The reference [4] is due to remarks from [5, p. 18] and [3, Theorem 3.3] with attribute to the unpublished paper of C. Stein. In the case of i.i.d. random variables, its proof can be found in [3, Theorem 3.3], [6, Theorem 12.8.1]. It is natural to expect that the relation (8) holds not only for i.i.d. random variables as in model (7), but in much more general cases. Following [6, Theorem 12.8.1], some particular analogs of formula (8) have already appeared for the cases of stationary Gaussian [7] and Poisson [8] random processes (with observation time $T$ instead of $n$).

Note that the relation (8) is essentially oriented to the case when the type-I and the type-II errors imply very different losses for us, and we are mainly interested in minimization of the type-II error probability $\beta={\mathbf{P}}\{H_{0}|H_{1}\}$. The case is quite popular in many applications (see, e.g., [7] and references therein). Also, Chernov [5, p. 17] wrote on such case: “It occasionally happens in practice that it is important to obtain $\beta$ very small, whereas a relatively large value of $\alpha$, like $.05$ or $.10$, is not disastrous”

In this paper, for the general model (1) some non-asymptotic generalization and strengthening of the relation (8) is derived. Such non-asymptotic results allowed to generalize the relation (8) for testing a simple hypothesis versus a composite one [8]. Also, such non-asymptotic results would allow essentially simplify similar results for Gaussian processes [7, 9].

1.1 Assumption

In order to simplify formulation of the results and to avoid pathological cases, we assume that the following assumption is satisfied:

I. Probability measures $\mathbf{P}$ and $\mathbf{Q}$ on a measurable space $(\cal X,{\cal B})$ are equivalent and Radon-Nikodim derivative $d{\mathbf{P}}/d{\mathbf{Q}}({\mathbf{x}})$ for them is defined, positive and finite for ${\mathbf{x}}\in{\cal X}$.

1.2 Main result

The main result of this paper is the following.

Theorem. If the assumption ${\bf I}$ holds, then the minimal possible $\beta(\alpha)$, $0<\alpha<1$, satisfies the bounds

[TABLE]

and

[TABLE]

where $\mu_{0}(\alpha)$ is the minimal value $\mu_{0}$, satisfying the relation

[TABLE]

With some loss of accuracy, we can simplify the upper bound (10), using some upper bound for the value $\mu_{0}(\alpha)$ (see (24) below). For example, we get

Corollary. The minimal possible $\beta(\alpha)$, $0<\alpha<1$, satisfies also the upper bound

[TABLE]

where

[TABLE]

Note that all bounds (9), (10) and (12) are pure analytical relations without any limiting operations. Also, for a fixed $\alpha$, both lower bound (9) and upper bound (12) are close to each other, if the value $r_{1}({\mathbf{P}},{\mathbf{Q}})$ is much smaller than $D({\mathbf{P}}||{\mathbf{Q}})$.

Remark 1. For the particular case of Poisson processes Theorem was proved in [8] (as Theorem 3).

Remark 2. Note that if in some asymptotic setting the Central Limit Theorem is applicable to the random variable $D({\mathbf{P}}||{\mathbf{Q}})-\ln(d{\mathbf{P}}/d{\mathbf{Q}})({\mathbf{x}})$, then the remaining term $r_{1}({\mathbf{P}},{\mathbf{Q}})$ in (12) has the optimal order (in $D({\mathbf{P}}||{\mathbf{Q}})$) for fixed $\alpha$ and large $D({\mathbf{P}}||{\mathbf{Q}})$ (see Example 3 below).

2 Proofs

2.1 Proof of Theorem

Derive first the lower bound (9). Without loss of generality and in order to avoid bulky formulas, note that using a natural randomization on the “bound” of a decision region ${\mathcal{D}}\in{\cal X}$, we have with ${\cal D}^{c}={\cal X}\setminus{\cal D}$ and $\beta=\beta({\cal D})$, $\alpha=\alpha({\cal D})$

[TABLE]

Since ${\mathbf{P}}({\cal D})=1-{\mathbf{P}}({\cal D}^{c})=1-\alpha$, then considering $d{\mathbf{P}}/(1-\alpha)$ as the probability distribution on ${\cal D}$, and using the inequality $\ln{\mathbf{E}}\xi\geq{\mathbf{E}}\ln\xi$, we have

[TABLE]

Since ${\mathbf{P}}({\cal D}^{c})=\alpha$, similarly to (15), the last term in the right-hand side of (15) gives

[TABLE]

Therefore, from (15) and (16) we have

[TABLE]

from where the lower bound (9) follows.

In order to prove the upper bound (10), we set a value $\mu>0$, and define the acceptance region in favor of ${\mathbf{P}}$

[TABLE]

Denote by $\alpha_{\mu}$ and $\beta_{\mu}$ the type-I and the type-II error probabilities for the acceptance region ${\cal A}_{\mu}$, respectively. Then by (18)

[TABLE]

Also

[TABLE]

where

[TABLE]

The best solution is to set $\mu$ such that $\alpha_{\mu}=\alpha$. Therefore, we define $\mu_{0}=\mu_{0}(\alpha)$ by the formula (11). Then we have

[TABLE]

and by (19)

[TABLE]

which proves the upper bound (10).

2.2 Proof of Corollary

In order to prove the upper bound (12), we simplify the relation (11), using the classic Chebyshev inequality

[TABLE]

where $z_{+}=\max\{0,z\}$. Then from (20) and (24)

[TABLE]

We set $\mu=\mu_{1}>0$ as follows (see (13))

[TABLE]

Then we get

[TABLE]

and by (19) and (26)

[TABLE]

which proves the upper bound (12).   $\Box$

3 Examples

Example 1 (Poisson processes). Denote by ${\mathbf{x}}_{T}=x_{{\mathbf{u}}_{T}}=\{x_{{\mathbf{u}}_{T}}(t),0\leq t\leq T\}$, the Poisson process with the intensity ${\mathbf{u}}_{T}=\{u_{T}(t),0\leq t\leq T\}$ Kutoyants (1984) and let ${\cal X}_{T}$ denote the observation space (i.e. ${\mathbf{x}}_{T}\in{\cal X}_{T}$). Consider testing a simple hypothesis $\mathcal{H}_{0}$ versus a simple alternative $\mathcal{H}_{1}$, based on observations ${\mathbf{x}}_{T}=\{{\mathbf{x}}_{T}(t),0\leq t\leq T\}$:

[TABLE]

where intensities ${\mathbf{p}}_{T}$ and ${\mathbf{q}}_{T}$ are given.

Poisson processes on $[0,T]$ with intensities ${\mathbf{p}}_{T}$ and ${\mathbf{q}}_{T}$ generate on $({\cal X}_{T},{\cal B}_{T})$ measures ${\mathbf{P}}_{{\mathbf{p}}_{T}}$ and ${\mathbf{P}}_{{\mathbf{q}}_{T}}$, respectively. The following formula for the Radon–Nikodim derivative holds (see, e.g., Kutoyants (1984))

[TABLE]

where for functions ${\mathbf{f}}=\{f(t),0\leq t\leq T\}$, ${\mathbf{s}}=\{s(t),0\leq t\leq T\}$ and Poisson process $x(t)$ we used the notation

[TABLE]

It is supposed that the right-hand side of (30) is well defined and therefore the measures ${\mathbf{P}}_{{\mathbf{p}}_{T}}$ and ${\mathbf{P}}_{{\mathbf{q}}_{T}}$ are equivalent (see, e.g., Kutoyants (1984)).

From (30) for any integrable function ${\mathbf{f}}_{T}=\{f_{T}(t),0\leq t\leq T\}$ the ”expectation” formula follows (see, e.g., Kutoyants (1984))

[TABLE]

where ${\mathbf{E}}_{{\mathbf{p}}_{T}}$ means expectation with respect to the measure ${\mathbf{P}}_{{\mathbf{p}}_{T}}$. In particular, from (32) we get

[TABLE]

Then, from (6), (30) and (33) we have

[TABLE]

For the remaining term $r_{1}({\mathbf{P}},{\mathbf{Q}})$ from (12)-(13) we get by (33) and (34)

[TABLE]

which gives an estimate to the convergence rate in (8) (provided the right-hand side of (35) is finite).

Example 2 (Gaussian processes). Note that by the Karhunen-Loéve expansion most of detection problems on continuous-time Gaussian processes can be reduced to the discrete-time case with Gaussian random vectors [11, Ch. VI]. Formally, such representation consists of infinite number of expansion terms, but in practice such infinite series can be well approximated by a finite summation (without essential loss for performance characteristics).

For that reason we consider testing of simple hypotheses $\mathcal{H}_{0}$ and $\mathcal{H}_{1}$, based on observation of Gaussian random vectors ${\mathbf{x}}_{n}\in{\mathbf{R}}^{n}$:

[TABLE]

where the sample $\boldsymbol{\xi}_{n}=(\xi_{1},\ldots,\xi_{n})$ represents “noise” and consists of independent and identically distributed Gaussian random variables with zero means and variances $1$. The stochastic “signal” $\boldsymbol{\eta}_{n}=(\eta_{1},\ldots,\eta_{n})$ is a Gaussian random vector with zero mean and covariance matrix $\mathbf{M}_{n}$ (which is symmetric and nonnegative-definite).

Denote by ${\mathbf{P}}_{n}$ and ${\mathbf{Q}}_{n}$ distributions of $\boldsymbol{\xi}_{n}$ and $\boldsymbol{\eta}_{n}$, respectively. Note that if $\det\mathbf{M}_{n}=0$, then distributions ${\mathbf{P}}_{n}$ and ${\mathbf{Q}}_{n}$ are orthogonal, and therefore, hypotheses $\mathcal{H}_{0}$ and $\mathcal{H}_{1}$ can be tested with $\alpha=\beta=0$.

Hence, we assume that $\det\mathbf{M}_{n}>0$, and then $\mathbf{M}_{n}$ is a symmetric and positive-definite matrix. Denote by $\lambda_{1},\ldots,\lambda_{n}$ the eigenvalues of $\mathbf{M}_{n}$ (all positive).

In order to simplify calculations, we reduce model (36) to the equivalent one with the diagonal matrix $\mathbf{M}_{n}$. Indeed, for the covariance matrix $\mathbf{M}_{n}$ there is the orthogonal matrix $\mathbf{T}_{n}$ and the diagonal matrix $\mathbf{\Lambda}_{n}$ such that $\mathbf{M}_{n}=\mathbf{T}_{n}\mathbf{\Lambda}_{n}\mathbf{T}_{n}^{\prime}$. Moreover, the diagonal matrix $\mathbf{\Lambda}_{n}=\mathbf{T}_{n}^{\prime}\mathbf{M}_{n}\mathbf{T}_{n}$ consists of the eigenvalues $\{\lambda_{i}\}$ of $\mathbf{M}_{n}$ [12, Ch. 4.7-9].

Also, for any orthogonal matrix $\mathbf{T}_{n}$, the distribution of $\mathbf{T}_{n}^{\prime}\boldsymbol{\xi}_{n}$ (for simple hypothesis $\mathcal{H}_{0}$ from (36)) remains the same as of $\boldsymbol{\xi}_{n}$. Therefore, multiplying both sides of (36) by $\mathbf{T}_{n}^{\prime}$, we reduce model (36) to the equivalent model with the diagonal matrix $\mathbf{\Lambda}_{n}$. In other words, model (36) takes the form

[TABLE]

Then we have

[TABLE]

and

[TABLE]

Also

[TABLE]

For simplicity, we assume that for a constant $C>0$ and large $n$ the following condition is satisfied

[TABLE]

Then, by (9), (12) and (39)-(41) we have

[TABLE]

which yields an estimate to the convergence rate in (8).

Example 3 (independent random variables). Consider a natural generalization of model (7), when observations ${\mathbf{x}}_{1}^{n}=(x_{1},\ldots,x_{n})\in{\mathbf{R}}^{n}$ are independent random variables with different distributions $\{{\mathbf{P}}_{i}\}$ or $\{{\mathbf{Q}}_{i}\}$, $i=1,\ldots,n$:

[TABLE]

Since

[TABLE]

the lower bound (9) takes the form

[TABLE]

We also have for the remaining term $r_{1}({\mathbf{P}}_{1}^{n},{\mathbf{Q}}_{1}^{n})$ from (13)

[TABLE]

For simplicity, we assume that for a constant $C>0$ and all sufficiently large $n$ the following condition is satisfied

[TABLE]

Then, by (46)-(47)

[TABLE]

Therefore, by (9) and (12) we have for $\ln\beta(\alpha)$

[TABLE]

which estimates the convergence rate in (8).

The author would like to thank Kutoyants Yu. A. and anonymous reviewers for useful discussions and constructive critical remarks, which improved the paper.