A Donsker-type Theorem for Log-likelihood Processes
Zhonggen Su, Hanchao Wang

TL;DR
This paper establishes a Donsker-type weak convergence result for the log-likelihood processes of a family of probability measures associated with a semimartingale, showing convergence to a Gaussian process under certain regularity conditions.
Contribution
It introduces a novel weak convergence theorem for log-likelihood processes in a semimartingale setting, extending classical results to more general stochastic processes.
Findings
Log-likelihood processes converge weakly to a Gaussian process
Convergence holds under conditions involving entropy and Hellinger processes
Results apply to a broad class of semimartingales
Abstract
Let be a complete stochastic basis, a semimartingale with predictable compensator . Consider a family of probability measures , where is an index set, , and denote the likelihood ratio process by . Under some regularity conditions in terms of logarithm entropy and Hellinger processes, we prove that converges weakly to a Gaussian process in as for each fixed .
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Methods and Inference · Stochastic processes and financial applications · Risk and Portfolio Optimization
A Donsker-type Theorem for Log-likelihood Processes
Zhonggen Su
School of Mathematical Science, Zhejiang University, Hangzhou, China.
and
Hanchao Wang
Zhongtai Securities Institute for Financial Studies, Shandong University, Jinan, 250100, China.
Abstract.
Let be a complete stochastic basis, a semimartingale with predictable compensator . Consider a family of probability measures , where is an index set, , and denote the likelihood ratio process by . Under some regularity conditions in terms of logarithm entropy and Hellinger processes, we prove that converges weakly to a Gaussian process in as for each fixed .
Key words and phrases:
Hellinger process of order zero; Log-likelihood process; Semimartinagle; Weak convergence
2010 Mathematics Subject Classification:
60F05, 60F17
1. Introduction and Preliminaries
The celebrated Donsker theorem is a functional extension of the central limit theorem in probability theory. Plenty of research on this topic has come out in the past decades. The reader is referred to classic books and papers like Dudley [3], Gine and Zinn [4], Ossiander [10], Andersen et. al [1], Liptser and Shiryaev [7], van der Geer [11], Billingsley [2], Jacod and Shiryaev [5] for both theoretical framework and wide applications. A primary purpose of the present paper is to establish a certain Donsker theorem for log-likelihood processes indexed by an arbitrary set. In this section, we first introduce some basic notions about log-likelihood processes and martingale representation property.
Throughout this paper, we follow the standard definitions and notations of martingale theory, which can be found in the book by Jacod and Shiryaev [5]. Let be a complete stochastic basis. Fix a semimartingale on it, and assume that all -martingales have a representation property relative to . Denote by the triplet the predictable characteristic of (associated to some bounded truncation function). More precisely, if denotes the jump of at time , then , where , is a special semimartingale, which can be uniquely divided into a bounded variation process and a local martingale process. The is a bounded variation process of . Let be the continuous local martingale part of , then
[TABLE]
Let be the jump measure of defined by
[TABLE]
where denotes the Dirac measure at point . The is the unique predictable compensator of (up to a -null set). Namely, is a predictable random measure such that for any predictable function111 Let , , where is a Borel -field on and a -field generated by all left continuous adapted processes on . The predictable function is a -measurable function on . , is a local martingale, where the is defined by
[TABLE]
(See Section 2.1 of Chapter 2 in Jacod and Shiryaev [5] for more details). Note the predictable quadratic variation is given by
[TABLE]
where
[TABLE]
and
[TABLE]
It follows from Corollary 1.19 of Chapter 2 in Jacod and Shiryaev [5] that is equivalent to the fact is a quasi-left continuous process222 A quasi-left continuous process is a càdlág adapted process such that for any increasing stoping times with limit , . Specially, for a process with independent increments, means this process has no fixed time of discontinuity 333 is called as the fixed time of discontinuity if .. Thus we may and do choose a good version of both and such that is the predictable projection of and . In particular,
[TABLE]
Now consider another probability measure such that
[TABLE]
which means that for any , . Define the likelihood ratio process
[TABLE]
It follows from Chapter III in Jacod and Shiryaev [5] that is a local martingale.
Since by assumption all -martingales have a representation property relative to , according to Theorem 5.19 of Chapter III in Jacod and Shiryayev [5], has the following representation: there is a predictable process and a nonnegative predictable function on such that
[TABLE]
Here
[TABLE]
and is a random set defined as follows
[TABLE]
[TABLE]
[TABLE]
[TABLE]
Note that and depend on . In fact, can be explicitly represented as follows. Let be a measure on where , , such that for all measurable nonnegative functions . Then is the conditional expectation of with respect to under , namely
[TABLE]
Define the log-likelihood process by
[TABLE]
This process has been a well-studied object in the context of both stochastic processes and statistical inferences. Obviously,
[TABLE]
Assume we are given a family of probability measures on , indexed by an arbitrary non-empty set , and assume
[TABLE]
for every and . We shall be mainly interested in the sequence of likelihood ratio processes . The main purpose of the paper is to establish a certain Donsker theorem for log-likelihood processes in as , where we denote by the space of bounded real-valued functions defined on .
It seems hard to develop directly an invariance principle for due to complicated structure. To the best of our knowledge, there are only few works in this area, such as Le Cam [6], Vostrikova [13] and so on. The reader can find some interesting results in Nishiyama [8] and [9], where is assumed to be very special continuous semimartingales and discrete time semimartingales respectively. It is a challenging problem to extend Nishiyama’s work to general setting. To attack such a problem, we shall combine stochastic calculus techniques and chaining arguments with the Kakutani-Hellinger distance for probability measures. In particular, we shall characterize the regularity of -valued log-likelihood processes in terms of the Kakutani-Hellinger distance and the Hellinger processes.
The rest of the paper is organized as follows. We will first make some necessary assumptions and then state our main results in the Section 2. The proof of main results is given in Section 3, which consists of several lemmas and two propositions.
2. Main Results
To state our main results, we need some more notations and make some technical assumptions. Start with the Kakutani-Hellinger distance between two probability measures and . Assume that is a third probability measure on such that
[TABLE]
Let
[TABLE]
and define the Kakutani-Hellinger distance by
[TABLE]
It is easy to check that is a metric in the space of probability measures and does not depend on the probability measure . Note
[TABLE]
For , call the Hellinger integral of order . We remark that as if .
Proceed to introduce the Hellinger processes. Assume that
[TABLE]
and define
[TABLE]
Then for each , there is a unique predictable increasing process , called the Hellinger process of order , such that
(i)
[TABLE]
(ii)
[TABLE]
(iii)
[TABLE]
where
[TABLE]
and
[TABLE]
One can extend the above Hellinger process to order zero and even to a general function. Given a function such that
[TABLE]
is bounded with convention and . Denote
[TABLE]
then there is a predictable increasing process, denoted by , such that
(i’)
[TABLE]
(ii’)
[TABLE]
(iii’)
[TABLE]
Call the Hellinger process of order 0 associated to . In particular, if
[TABLE]
then we simply call the Hellinger process of order 0.
In general, it is rather complicated to compute . However, we fortunately have the following explicit formula in the special case :
[TABLE]
In particular,
[TABLE]
[TABLE]
Our technical assumptions mainly involve three aspects: the predictable envelope of , the Kakutani-Hellinger distance between probability measures , and the size of index set .
For every , denote the essence supremum . This is the predictable envelope of used in Definitions 2.1 and 2.3 of Nishiyama [9].
Assumption 1**.**
For any , , and . Moreover, attains their predictable envelope for every , namely, there is a such that .
Assumption 2**.**
For every , as
[TABLE]
where .
There is a nonnegative definite continuous function on , such that as ,
[TABLE]
and for every ,
[TABLE]
Let be an arbitrary set, a positive rational number. , is called a decreasing series of finite partitions (DFP) of if
(i) each is a finite partition of , namely
[TABLE]
(ii) and ;
(iii) as .
Given a , the -entropy is defined by
[TABLE]
Assumption 3**.**
There exists a decreasing series of finite partitions, , of such that as
[TABLE]
and
[TABLE]
where
[TABLE]
We are now ready to state our main result as follows.
Theorem 2.1**.**
Under Assumptions 1, 2 and 3, we have
[TABLE]
where stands for a Gaussian element in , each -dimensional marginal is a normal random vector with mean
[TABLE]
and covariance structure
[TABLE]
The proof will be given in Section 2. For sake of comparison, we review an earlier result due to Nishiyama [9] in the discrete time case. Let be a discrete time stochastic basis, and a family of probability measures on , such that
[TABLE]
Define
[TABLE]
and
[TABLE]
Nishiyama [9] studied weak convergence for log-likelihood processes in and obtained a similar result to (2.13) under some integrability assumptions involving ’s and entropy conditions. More specifically, assume
(i) for every
[TABLE]
(ii) there exists a decreasing series of finite partitions, , of such that
[TABLE]
and
[TABLE]
(iii) there is a such that
[TABLE]
and for
[TABLE]
Then
[TABLE]
where stands for a Gaussian element in , each -dimensional marginal is a normal random vector with mean
[TABLE]
and covariance structure
[TABLE]
To conclude the Introduction, two more remarks are given .
Remark 2.2*.*
Observe in the discrete time case the Hellinger process can be computed as follows.
[TABLE]
and
[TABLE]
Thus there is to some extent a similarity between our assumptions in Theorem 2.1 and Nishiyama’s assumptions. However, it seems neater to use the Hellinger processes in continuous time case.
The integrability condition (Assumption 3) of partitioning entropy plays an important role in the proof of Theorem 2.1. It is possible to use the metric entropy condition, but we need to introduce a suitable pseudo-metric in the index set . The Hellinger processes would also be very likely a good candidate.
Remark 2.3*.*
It is rather interesting to consider the limiting behavior of the process in . To this end, we need to establish a tightness criterion in the space . This is more complicate, and will be left to the future work.
3. Proofs
Let us start with a decomposition. Observe that
[TABLE]
It is easy to see
[TABLE]
and so we have
[TABLE]
Let be the jump measure of defined by
[TABLE]
and the corresponding predictable compensator. Then for any predictable function ,
[TABLE]
and so by the fact that is the predictable projection of ,
[TABLE]
Given a positive number , consider the truncation function
[TABLE]
and define
[TABLE]
Thus combined together, we easily have a canonical decomposition
[TABLE]
For simplicity of writing, let
[TABLE]
[TABLE]
[TABLE]
Thus we have
[TABLE]
The proof of Theorem 2.1 will consist of a series of lemmas and propositions.
Lemma 3.1**.**
Under Assumptions 1, 2 and 3, we have for each and , as
[TABLE]
Proof.
Set
[TABLE]
Then
[TABLE]
Also, for any
[TABLE]
By the Lenglart domination property (see page 35 of Jacod and Shiryaev [5]),
[TABLE]
Note for there is a positive constant such that for any
[TABLE]
so
[TABLE]
On the other hand, for each
[TABLE]
Again, by the Lenglart domination property, it follows for any
[TABLE]
Letting and then , we have
[TABLE]
In combination, we have proved the desired statement. ∎
Lemma 3.2**.**
Under Assumptions 1, 2 and 3, we have for each , as
[TABLE]
[TABLE]
[TABLE]
Consequently,
[TABLE]
Proof.
Obviously, for any
[TABLE]
Also, by Assumption 2
[TABLE]
The desired (3.8) holds.
Observe an elementary inequality: for any , there is a positive constant such that
[TABLE]
Then it follows
[TABLE]
[TABLE]
[TABLE]
Thus under the Assumption 2, we have by letting and then
[TABLE]
For (3.10), note
[TABLE]
where is as in (3.6). Then it easily follows
[TABLE]
[TABLE]
[TABLE]
Thus under the Assumption 2, we have by first letting and then
[TABLE]
The proof is now complete. ∎
Lemma 3.3**.**
Under Assumptions 1, 2 and 3, we have for each and , as
[TABLE]
Proof.
First, observe the quadratic variation of is given by
[TABLE]
We shall prove that converges in probability to zero below. Note there is a such that for any
[TABLE]
where . Thus it follows for any
[TABLE]
Hence letting and then immediately yields
[TABLE]
A similar argument shows
[TABLE]
Combined together, we have the desired statement. ∎
Lemma 3.4**.**
Under Assumptions 1, 2 and 3, we have for each and , as
[TABLE]
Proof.
Note there is a such that for any
[TABLE]
where . Thus it follows for any
[TABLE]
This implies
[TABLE]
Since it was proved , then we have
[TABLE]
Similarly, we have
[TABLE]
Combined together, the proof is complete. ∎
Proposition 3.5**.**
Under Assumptions 1, 2 and 3, every finite-dimensional marginal of converges weakly.
Proof.
[TABLE]
has non-degenerate limiting finite-dimensional marginal laws, and the other part of asymptotically vanish.
For every , the process
[TABLE]
is a continuous semimartingale. Its predictable characterstics are
[TABLE]
[TABLE]
By Lemma 3.2 and Assumption 2, there is a non-decreasing continuous function , such that ,
[TABLE]
[TABLE]
for every .
The proposition is now concluded by Theorem VIII.3.6 of Jacod and Shiryaev [5]. ∎
Next we turn to verifying uniform tightness.
Lemma 3.6**.**
Under Assumptions 1, 2 and 3, we have for each , as
[TABLE]
Proof.
Recall
[TABLE]
Let us prove
[TABLE]
and
[TABLE]
For (3.19), note
[TABLE]
Thus we need only to prove
[TABLE]
and
[TABLE]
Let us first look at (3.21). Set
[TABLE]
Then
[TABLE]
For any ,
[TABLE]
Note if . Then
[TABLE]
Recall the definition of and Assumptions 1 and 2, we can obtain (3.21). The proofs of and are similar. ∎
Lemma 3.7**.**
Under Assumptions 1, 2 and 3, for any , there is a and a partition such that
[TABLE]
Proof.
Let us fix . First, note
[TABLE]
Let
[TABLE]
and
[TABLE]
We shall treat and separately below. Let us only focus on the since the is similar and simpler.
According to Assumption 3, there is a sufficiently large positive finite constant such that
[TABLE]
Thus we only need to condition on the event . In particular, we shall prove
[TABLE]
Assuming (3.28), we can take so small that
[TABLE]
from which it follows by the Markov inequality
[TABLE]
This in turn together with (3.27) implies
[TABLE]
It remains to prove (3.28). For every integer , construct a nested refinement partition of , and then choose an element from each partitioning set in such a way that
[TABLE]
For every and each , define and whenever . Obviously, . Define
[TABLE]
Note . Set
[TABLE]
and
[TABLE]
and for
[TABLE]
[TABLE]
It is easy to see
[TABLE]
and for each
[TABLE]
Hence it follows for any
[TABLE]
Note if , and so we have
[TABLE]
It is now enough to show
[TABLE]
We have the following identity
[TABLE]
Denote
[TABLE]
We shall only establish (3.32) for since the other three terms in RHS of (3) can be similarly treated.
Obviously, is a local martingale, and
[TABLE]
On the other hand, the predictable quadratic variation of satisfies
[TABLE]
By an elementary calculation, for any there is a constant such that
[TABLE]
whenever . Then it follows
[TABLE]
By Bernstein-Freedman’s inequality (see Lemma 3.2 of Nishiyama [8]) for local martingale with bounded jumps, it follows for ,
[TABLE]
This, in turn together with Lemma 2.2.10 of van der Vaart and Wellner [12], yields
[TABLE]
and
[TABLE]
Thus (3.28) is obtained, and so complete the proof. ∎
Lemma 3.8**.**
Under Assumptions 1, 2 and 3, for any , there is a and a partition such that
[TABLE]
Proof.
Recall
[TABLE]
It is enough to prove the following two statements
[TABLE]
and
[TABLE]
We shall concentrate on proving (3.34) below since (3.35) is similar. The proof is completely similar to that of Lemma 3.7 with some minor modifications. For every integer , choose an element from each partitioning set in such a way that
[TABLE]
and define and whenever . Note
[TABLE]
then a main step is to prove
[TABLE]
To this end, for , set
[TABLE]
Obviously, . Define
[TABLE]
[TABLE]
[TABLE]
where is as in (3.31).
Note we have the following identity
[TABLE]
and
[TABLE]
In addition, it is easy to see
[TABLE]
by Schwarz’s inequality. Thus
[TABLE]
Thus, (3.34) is proved. We complete the proof of this lemma. ∎
We can obtain the following proposition by Lemmas 3.6 - 3.8.
Proposition 3.9**.**
Under Assumptions 1, 2 and 3, for any , there is a and a partition such that
[TABLE]
The proof of Theorem 2.1. Proposition 3.9 implies the asymptotic equicontinuity of , and the asymptotic marginal distribution of is obtained by Proposition 3.5. Then we can obtain Theorem 2.1 by these two propositions and Theorem 1.1 in Nishiyama [9].
Acknowledgments
The authors would like to thank the anonymous referees and the Associate Editor for careful reading and constructive comments. This work was supported by the National Natural Science Foundation of China (No.11371317, 11701331, 11731012, 11871425) , Fundamental Research Funds for Central Universities, Shandong Provincial Natural Science Foundation (No. ZR2017QA007) and Young Scholars Program of Shandong University.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Andersen, N., Giné, E., Ossiander, M. and Zinn,J. The central limit theorem and the law of iterated logarithm for empirical processes under local conditions. Probab. Theory Related Fields , 77 , 271-305, (1988).
- 2[2] Billingsley, P. Convergence of Probability Measures , 2nd ed. John Wiley and Sons, (1999).
- 3[3] Dudley, R. Central limit theorems for empirical measures. Ann. Probab. , 6 , 899-929, (1978).
- 4[4] Giné, E. and Zinn,J. Some limit theorems for empirical processes. Ann. Probab. , 12 , 929-989, (1984).
- 5[5] Jacod, J. and Shiryaev, A. Limit Theorems for Stochastic Processes , 2nd ed. Grundlehren der Mathematischen Wissenschaften 288 . Springer, Berlin, (2003).
- 6[6] Le Cam, L. Locally asymptotically normal families of distributions. Univ. of Calif. Publ. Stat. , 3 , 27-98, (1960).
- 7[7] Liptser, R. and Shiryaev, A. Theory of Martingale , Kluwer ascdemic phblishers, Dordrecht, (1989).
- 8[8] Nishiyama, Y. Some central limit theorems for l ∞ − limit-from superscript 𝑙 l^{\infty}- valued semimartingales and their applications. Probab. Theory Related Fields , 108 , 459-494, (1997).
