Limit theorems for pseudo sum of discrete distributions
Ivan Alexeev, Ignat Melnikov, Artem Uglovski

TL;DR
This paper introduces associative Look-Up Tables to accurately determine pseudo sums of discrete distributions, characterizes the limit distributions in pseudo-summation schemes, and explores conditions for domain attraction of stable distributions.
Contribution
It presents a novel approach using associative Look-Up Tables for pseudo sums and fully characterizes limit distributions and domain attraction conditions.
Findings
Set of limit distributions in pseudo-summation schemes described
Stable and infinitely divisible distributions characterized
Conditions for domain of attraction of stable distributions provided
Abstract
In this article we introduce associative Look-Up Tables. With their help, pseudo sums are correctly determined. The set of limit distributions in a pseudo-summation scheme of i.i.d. random variables is described. Also, two special cases that are similar to the classical sum and maximum operations are considered. In both situations, the set of stable distributions and the set of infinity divisible distributions are fully described. In addition, necessary and sufficient conditions for random variable to belong to the domain of attraction of stable random variable are introduced.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsProbability and Risk Models
Limit theorems for pseudo sum of discrete distributions.††thanks: The work was supported by the Russian Science Foundation (grant No. 22-21-00016).
I. A. Alexeev 111Institute for Information Transmission Problems of Russian Academy of Sciences, Bolshoy Karetny per. 19, build.1, Moscow and St. Petersburg Department of Steklov Mathematical Institute of Russian Academy of Sciences, 27 Fontanka, St. Petersburg, Russia, email: [email protected], I.A. Melnikov222Institute for Information Transmission Problems of Russian Academy of Sciences, Bolshoy Karetny per. 19, build.1, Moscow, email: [email protected], A.Y.Uglovski333Institute for Information Transmission Problems of Russian Academy of Sciences, Bolshoy Karetny per. 19, build.1, Moscow, email: [email protected]
Annotation
In this article we introduce associative Look-Up Tables. With their help, pseudo sums are correctly determined. The set of limit distributions in a pseudo-summation scheme of i.i.d. random variables is described. Also, two special cases that are similar to the classical sum and maximum operations are considered. In both situations, the set of stable distributions and the set of infinity divisible distributions are fully described. In addition, necessary and sufficient conditions for random variable to belong to the domain of attraction of stable random variable are introduced.
Keywords and phrases: limit theorems, discrete distributions, stable distributions, infinite divisibility.
1 Introduction
The present paper is devoted to the study of discrete stable distributions in the case when classical summation is changed to some associative operation.
In the early 1930s P. Lévy described the set of limit distributions in a summation scheme of independent identically distributed (i.i.d.) random variables with some positive normalization and real-valued centering (see [5]). He proved that if for some and , we have
[TABLE]
then has to be stable. Here and after, denotes the convergence in distribution.
Recall that, a random variable is called stable if for every , there exist and such that
[TABLE]
where , are independent copies of and denotes the equality in distribution. The class of classical one-dimensional stable distributions is well studied.
It has been noted that the sum operation does not play such a significant role in Lévi’s results. To a greater extent, an associative, commutative and reversible operation is required. For example, similar results exist for maximum stable laws (see [10]). More precisely, if for some , we have
[TABLE]
then has to be max-stable.
Both stable and max-stable distributions are well studied and fully characterized (see [2], [4], [5], [8], [10], [11]).
The motivation for this article was the results on stable laws in all their understandings and the theory of Low-Density Parity Check codes. Namely, in standard Information Bottleneck decoder so called Look-Up Tables are used. Firstly, quantize data is used when transmitting a message, that is, only a finite number of values are transmitted and some special sum operation is used when updating messages. This article is devoted to such sum operations and limit theorems with them.
Similar limit theorems have already been studied before. In the case when the operation is associative, then the set with the operation is a semigroup. Limit theorems with a semigroup operation were studied in the [7]. The case of a non-associative operation was studied in the articles [6], [12] from the point of view of a Markov chain with a finite number of values.
In section 2 associative Look-Up Tables are introduced and the most general properties are proved. In particular, the set of limit distributions in a pseudo-summation scheme of i.i.d. random variables is described. Section 3 is devoted to two special cases that are similar to the classical sum and maximum operations. In both situations, the set of stable distributions and the set of infinity divisible distributions are fully described. Also, necessary and sufficient conditions for random variable to belong to the domain of attraction of stable random variable are introduced.
2 General results
Let , be a finite set of distinct real values, i.e. , , for . Every function can be presented as a matrix , where for all .
Let assume that for all the following condition holds:
[TABLE]
It follows that function (or matrix ) can correctly denote a pseudo-summation operation:
[TABLE]
Condition 1 yields the associativity of a pseudo-summation, that is
[TABLE]
Let us note that such an operation is commonly used in the coding theory, namely, in the standard Information Bottleneck decoder for Low-Density Parity Check codes (for further information see, for instance, [3]). The matrix is called Look-Up Table. Further, this is the name that will be used. Also, if for condition 1 holds, then will be called associative Look-Up Table.
For each associative Look-Up Table, similarly to the classical real case, stable random variables can be determined for the pseudo-summation.
Definition 1
A random variable taking values in is called stable if , where , are independent copies of .
One can easily show that is stable if for every the sum , where are independent copies of .
The set of stable distributions directly depends on the Look-Up Table . Thus, we will write that random variable is -stable in order to highlight with respect to which operation the random variable is stable.
Similar to the classic sum, stable distributions pays the crucial role in the limit theorems for pseudo-summation.
Theorem 1
Let , be a sequence of i.i.d. random variables taking values in and let be an associative Look-Up Table. If for some random variable we have
[TABLE]
then is -stable.
Proof. The proof is almost no different from the real case. Since is an associative Look-Up Table, then
[TABLE]
where , and are independent copies of . Since the left hand-side of (2) converges to , then , which means that is -stable.
Let assume that there exists such that for every the equation has a unique solution . In other words, on some subset a left pseudo-subtraction can be defined. If , are independent random variables and uniformly distributed on the set , then for all we have
[TABLE]
where is the number of elements in .
Since for every there exists unique such that , then and, hence, is also a uniformly distributed random variable. It follows that if on some subset a left pseudo-subtraction can be defined, then random variable that uniformly distributed on is -stable.
Let assume that , . Then if , it follows that the degenerate law at point is -stable.
On the other hand, if degenerate law at point is -stable, then
[TABLE]
It follows that and, consequently, the degenerate law at point is -stable if and only if .
In addition to associativity, we introduce two more conditions. Let assume that and that for an associative Look-Up Table there exist two matrices and such that , for every , and for every we have
[TABLE]
where is an operation generated by .
Lemma 1
If Look-Up Table satisfy (3), then for every independent random variables , taking values in we have
[TABLE]
where \mathcal{L}(X)=\bigr{(}\text{Pr}(X=i)\bigr{)}, is a vector in corresponding to the distribution of , by we denote the component-by-component multiplication.
Proof. For simplicity, let , , and . Then , , and , where denote a standard basis in . Then
[TABLE]
where is a -th column of a matrix .
Similarly one can show that .
Lemma (1) shows that if Look-Up Table satisfy condition (3), then there exists a characteristic function. Indeed, if is a distribution of random variable , then uniquely defines and for any other distribution we have
[TABLE]
where is a distribution of pseudo sum of independent random variables with distributions and respectively. In other words, such a pair of matrices diagonalizes the pseudo-convolution operator.
Let us note that not all Look-Up Tables satisfy (3) and accordingly (4). For instance, if for all , then for all . Since , then for all independent random variables , we have and
[TABLE]
Then for any random variable the vector V\bigr{(}\mathcal{L}(X)\bigr{)}\in\{0,1\}^{N}. It follows that either or for all . The latter instantly leads to a contradiction with the invertibility of the matrix . Then we have that for all independent random variables and we have U\bigr{(}\mathcal{L}(X)\bigr{)}=U\bigr{(}\mathcal{L}(Y)\bigr{)}. is an invertable matrix, then , which leads to a contradiction.
It is trivial that if and only if , where , are the distributions of , respectively. Also, if , then .
The following definition is a trivial generalization of the classical one.
Definition 2
A random variable taking values in is said to belong to the domain of attraction of a random variable if
[TABLE]
The following theorem shows the necessary condition for random variable to belong to the domain of attraction of another random variable .
Theorem 2
Let be associative Look-Up Table satisfying (3). If a random variable with distribution belongs to the domain of attraction of the random variable with distribution , then , i.e. .
Proof. Let belongs to the domain of attraction of random variable . Then
[TABLE]
It follows that . Since defines uniquely, then or .
Corollary 1
Let be associative Look-Up Table satisfying (3) and for some . If a random variable belongs to the domain of attraction of the degenerate law at point , then .
3 Special cases
In this paragraph two special cases will be considered. As previously, let us assume that .
A) Firstly, let us consider the following Look-Up Table:
[TABLE]
where is some permutation, is a remainder of the division.
Theorem 3
The operation given by the formula (5) satisfy four following conditions:
- (a)
Associativity – for all ; 2. (b)
Commutativity – for all ; 3. (c)
For all the equation has a unique solution . 4. (d)
There exists such that for all .
Proof. First, let us show that it is sufficient to prove that for all we have
[TABLE]
If (6) holds, then \Bigr{(}\chi,\oplus\Bigr{)} is a group isomorphic to the \Bigr{(}\{e^{i\tfrac{2\pi k}{N}}\}_{k=1}^{N-1},\cdot\Bigr{)}. It means that , as the , satisfy all four conditions .
Let us now prove (6). From (5) it follows that is equivalent to s(z)=\bigr{(}s(x)+s(y)\bigr{)}\%N. It means that is either or and, consequently,
[TABLE]
Let us now consider the right hand-side of the (6). It is equivalent to the following: there exists such that . Since , then either or , which is equivalent to the left hand-side of the (6).
Theorem 3 also shows that for random variable taking values in with pseudo-summation (5) there exists a classic characteristic function. Indeed, function
[TABLE]
uniquely defines the random variable and if and are independent, then
[TABLE]
In terms of matrices and from (3) we have the following:
[TABLE]
Without loss of generality, we can consider only identical permutation. Indeed, characteristic function of equals the following:
[TABLE]
Further, all the results will be formulated only for the identical permutation. The rest of the results are obtained by replacing the variable .
Similarly to the general case, we first consider stable distributions. To highlight this particular case, we will write -stable random variables instead of -stable ones.
Theorem 4
Random variable is -stable if and only if it is either degenerate at point [math] or there exists a divisor of a number such that , where is a uniformly distributed random variable on the set .
Proof. Random variable is -stable if and only if for every we have
[TABLE]
First of all, let us notice that a degenerate law at point [math] satisfy this condition. If is a divisor of a number and is a uniformly distributed random variable on the set , where . Then for all , , the characteristic function of random variable equals
[TABLE]
If , then and , otherwise (see [1, Theorem 6.10]).
Let us show the opposite. Form (7) it follows that for every the function is either [math] or . If , then can only be degenerate law at point [math].
Now let us assume that for some function . Let be such a number that and for all the function . Then
[TABLE]
For (8) to be equality it is necessary and sufficient that , i.e.
[TABLE]
It follows that, first of all, is a divisor of a number and, secondly, there exists a random variable that takes values in the set such that . Since for all characteristic function , then for all characteristic function . It is [1, Theorem 6.10] that states that has uniform distribution on and concludes the proof.
Let us note that Theorem 4 possibly can be proved earlier. However, the authors failed to find the original proof.
Corollary 2
If is a prime number, then random variable is -stable if and only if it is either degenerate at point [math] or uniformly distributed on .
The following theorem provides the necessary and sufficient conditions for random variable to belong to the domain of attraction of each stable law.
Theorem 5
Let be a divisor of a number , , and random variable has uniform distribution on the set . Random variable belongs to the domain of attraction of if and only if there exists random variable such that and for all and we have
[TABLE]
Proof. It has already been shown that characteristic function of equals
[TABLE]
It follows that belongs to the domain of attraction of if and only if for every we have
[TABLE]
In particular, \lim\limits_{m\to\infty}\Bigr{(}f_{X}(M)\Bigr{)}^{m}=1 and, as a consequence, . It has already been shown at the proof of the Theorem 5 that there exists a random variable taking values in such that .
Let , be the sequence of independent copies of , then
[TABLE]
if and only if
[TABLE]
Now, without loss of generality we can assume that . Then a random variable belongs to the domain of attraction of random variable with uniform distribution on if and only if for all we have
[TABLE]
One can see that (9) holds if and only if for all we have . Let assume that there exists and such that
[TABLE]
It follows that does not belong to the domain of attraction of if and only if there exists and such that
[TABLE]
As noted earlier, \Bigr{(}\chi,\oplus\Bigr{)} is a group, moreover, it is locally compact and Abelian. For such groups, a complete classification of infinitely divisible distributions in terms of their characteristic functions is known (see [9]). Recall that a random variable is called infinitely divisible if for every there exists a random variable such that
[TABLE]
where are independent copies of , .
Theorem 6
For any infinitely divisible random variable there exists , a divisor of , and a random variable such that
[TABLE]
where has a uniform distribution on the set , , are independent copies of , and has the Poisson distribution with intensity .
Proof. From [9, Theorem 7.1] the random variable is infinity divisible if and only if its characteristic function has the following representation
[TABLE]
where is a characteristic function of with some , , , for all the function satisfy the following condition:
[TABLE]
To prove the Theorem 6 it is sufficient to show that for all . Indeed, corresponds to the random variable , i\frac{2\pi}{N}t\bigr{(}a-\sum_{k=0}^{N-1}c_{k}k\bigr{)} corresponds to the shift, and finally, if and , , then
[TABLE]
Since and , then
[TABLE]
Now let us show that for all . In (10) let suppose that , then . It follows that . Let us prove that for every there exists an integer such that and, moreover, . For and it is obvious that and .
In (10) let suppose that , , then
[TABLE]
From the induction step, it follows that
[TABLE]
One can see that and from the induction step we have , which leads to the following inequality .
Hence, it has been proved that for any there exists an integer such that and .
In (10) let . Then
[TABLE]
Similarly, let . Then
[TABLE]
From (11) and (12) one can see that for any we have . If , then . However, for it has been proved that , which follows that and, consequently, for all .
B) Another special case of a Look-Up Table is the maximum value, that is
[TABLE]
It is obvious that is an associative and commutative operation. Also, for every we have . In Section 2 it has been proved that degenerate law at point is stable for all . In this case, similar to the real-valued case, we will write that degenerate laws are max-stable.
Let us show that there is no other max-stable laws. If is max-stable, then for every we have
[TABLE]
where , are independent copies of .
Since , then for every we have
[TABLE]
It means that for every the probability \text{Pr}\bigr{(}\xi\leqslant x\bigr{)} is either 0 or 1. Since is a non-decreasing function, then is degenerate random variable. These arguments entail the following theorem.
Theorem 7
Random variable is max-stable if and only if it is degenerate at some point .
Theorem 2 yields the necessary and sufficient conditions for random variable to belong to the domain of attraction of a degenerative law at point . Namely, the following theorem is true.
Theorem 8
Random variable belongs to the domain of attraction of a degenerate law at point if and only if and .
Proof. From Theorem 2 it follows that if belongs to the domain of attraction of a degenerate law at point , then . If , then for all , which entails a contradiction.
Let now assume that and . Then for we have
[TABLE]
Since , then . It follows that converges to degenerative law at point .
Similar to the previous case, let us study infinitely divisible distributions. Let assume that random variable with the distribution function is infinitely divisible. It means that for every there exists a random variable with the distribution function such that
[TABLE]
where are independent copies of .
Thus, in terms of distribution functions, distribution function is infinitely divisible if and only if for every there exists a distribution function such that
[TABLE]
Let us fix an arbitrary distribution function . It is piece-wise constant function with jumps in points , then for all the function F_{n}(x)=\bigr{(}F(x)\bigr{)}^{1/n} is also a non-decreasing piece-wise constant function with jumps in points and
[TABLE]
It means that is a distribution function of a discrete law on . Hence, if , then any random variable is infinitely divisible.
Let us note that in this case Look-Up Table also satisfy (3) with matrices
[TABLE]
where if and 0 otherwise.
Acknowledgments
The work of I. A. Alexeev was supported in part by the Moebius Contest Foundation for Young Scientists.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1[1] Tom M. Apostol, Introduction to Analytic Number Theory , Springer New York, NY, 1976.
- 2[2] B. V. Gnedenko and A. N. Kolmogorov, Limit Distributions for Sums of Independent Random Variables , Addison-Wesley, Cambridge, MA, 1954.
- 3[3] X. He, K. Cai and Z. Mei, On Mutual Information-Maximizing Quantized Belief Propagation Decoding of LDPC Codes , 2019 IEEE Global Communications Conference (GLOBECOM), Waikoloa, HI, USA, 2019, pp. 1-6.
- 4[4] I. A. Ibragimov and Yu. V. Linnik, Independent and Stationary Sequences of Random Variables , Wolters-Noordhoff Publishing, Groningen, 1971.
- 5[5] P. Lévy Sur les intégrales dont les éléments sont des variables aléatoires indépendantes , Annali della Scuola Normale Superiore di Pisa - Classe di Scienze, Scuola normale superiore, 2e série, 3 , 1934 , pp.337-366.
- 6[6] S. Markovski, D. Gligoroski, V. Bakeva, Quasigroup string processing. I , Makedon. Acad. Nauk. Umet. Oddel. Mat.-Tehn. Nauk. Prilozi, 20 (1-2), 1999, pp. 13-28.
- 7[7] P. Martin-L?f, Probability theory on discrete semigroups , Z. Wahrscheinlichkeitstheorie verw Gebiete 4 , 1965, pp. 78-102.
- 8[8] M. M. Meerschaert and H.-P. Scheffler, Limit Distributions for Sums of Independent Random Vectors. Heavy Tails in Theory and Practice , Wiley Ser. Probab. Statist. Probab. Statist., John Wiley & Sons, New York, 2001.
