A version of Herbert A. Simon's model with slowly fading memory and its connections to branching processes

TL;DR

This paper introduces a recursive word-generation model inspired by Herbert A. Simon's work, analyzing its long-term behavior and connection to branching processes, revealing power-law and exponential decay regimes based on parameters.

Contribution

It extends Simon's model by incorporating slowly fading memory and establishes links to branching processes, providing new insights into word frequency distributions.

Findings

Proportion of words with exact repetitions converges as string length grows.

Power-law decay in word frequency distribution when certain parameter conditions are met.

Exponential decay occurs in the distribution under different parameter regimes.

Abstract

Construct recursively a long string of words w1. .. wn, such that at each step k, w k+1 is a new word with a fixed probability p $\in$ (0, 1), and repeats some preceding word with complementary probability 1 -- p. More precisely, given a repetition occurs, w k+1 repeats the j-th word with probability proportional to j $\alpha$ for j = 1,. .. , k. We show that the proportion of distinct words occurring exactly times converges as the length n of the string goes to infinity to some probability mass function in the variable $\ge$ 1, whose tail decays as a power function when 1 -- p > $\alpha$/(1 + $\alpha$), and exponentially fast when 1 -- p < $\alpha$/(1 + $\alpha$).

Equations

$$\mathbb{P}(J=j)=\frac{q(j,k)}{{\sum }_{i=1}^{k}q(i,k)},j=1,\dots ,k.$$

$$q(i,k)=q(i)=i^{\alpha },\text{for all 1≤i≤k}$$

$$\underset{n\to \infty }{\lim }\frac{{\nu }_n(j)}{np}=\rho \mathrm{B}(j,\rho +1)\text{a.s. for all j≥1,}$$

$$\underset{n\to \infty }{\lim }\frac{{\nu }_n(\ell )}{np}=\phi (\ell )\text{in probability,}$$

$$\sum _{j>k}\phi (j)\sim C{k}^{-1/(\bar{p}(1+\alpha )-\alpha )}\text{as }k\to \infty$$

$$\sum _{j=1}^{\infty }{\mathrm{e}}^{cj}\phi (j)<\infty .$$

$$N_j(n)=\mathrm{Card}\{1\le k\le n:w_k=w_j\}$$

$$A_j(n)=n^{-\alpha }\sum _{k=1}^n{k}^{\alpha }\text{mathbbm}{1}_{\{w_k=w_j\}}.$$

$$N_j(n)=\mathrm{Card}\{0\le k\le n-1:A_j(k)<A_j(k+1)\}.$$

$$\underset{\epsilon \to 0+}{\lim }\frac{E_u\left(f(A(t+\epsilon ))\mid A(t)=a\right)-f(a)}{\epsilon }$$

$$N(t)=\mathrm{Card}\{0<s\le t:A(t)=A(t-)+1\}.$$

$$(A_{j(n)}(\lfloor tn\rfloor ),N_{j(n)}(\lfloor tn\rfloor ){)}_{t\ge 0}$$

$$(A_{j(n)}(\lfloor tn\rfloor ),N_{j(n)}(\lfloor tn\rfloor ){)}_{t\ge 0}\text{and}(A_{k(n)}(\lfloor tn\rfloor ),N_{k(n)}(\lfloor tn\rfloor ){)}_{t\ge 0}$$

$$\mathbb{P}\left(A_j(\ell +1)=\frac{{\ell }^{\alpha }a}{(\ell +1{)}^{\alpha }}+1,N_j(\ell +1)=k+1\mid A_j(\ell )=a,N_j(\ell )=k\right)$$

$$\mathbb{P}\left(A_j(\ell +1)=\frac{{\ell }^{\alpha }a}{(\ell +1{)}^{\alpha }},N_j(\ell +1)=k\mid A_j(\ell )=a,N_j(\ell )=k\right)$$

$$s_{\ell }=\sum _{i=1}^{\ell }{i}^{\alpha }\sim {\ell }^{\alpha +1}/(\alpha +1).$$

$$\underset{n\to \infty }{\lim }$$

$$\mathcal{G}f(x)=-bx{f}^{\prime }(x)+x(f(x+1)-f(x)),$$

$$b=\frac{\alpha }{\bar{p}(\alpha +1)}.$$

$$\{\begin{array}{cc}0& \text{for }t<u,\\ Z(\alpha b^{-1}\ln (t/u))& \text{for }t\ge u,\end{array}$$

$${\xi }_t={\eta }_t-bt\text{ for every }t\ge 0.$$

$$\zeta =\inf \{t>0:{\xi }_t=0\}$$

$$T(t)={\int }_0^{t}Z(s)\mathrm{d}s,$$

$$B(t)=Z(t)+b{\int }_0^{t}Z(s)\mathrm{d}s,$$

$$\{\begin{array}{cc}0& \text{for }t<u,\\ B(\alpha b^{-1}\ln (t/u)))& \text{for }t\ge u,\end{array}$$

$$W_t={\mathrm{e}}^{-(1-b)t}Z(t),t\ge 0$$

$$\mathbb{E}(Z(t{)}^k)\le c_k{\mathrm{e}}^{k(1-b)t}\text{for all }t\ge 0.$$

$$\mathcal{G}f(x)=-\ell b{x}^{\ell }+x\sum _{j=0}^{\ell -1}\left(\genfrac{}{}{0ex}{}{\ell }{j}\right)x^j=\ell (1-b)x^{\ell }+\sum _{j=0}^{\ell -2}\left(\genfrac{}{}{0ex}{}{\ell }{j}\right)x^{j+1}.$$

$$\frac{\mathrm{d}}{\mathrm{d}t}\mathbb{E}(f(Z(t)))=\mathbb{E}(\mathcal{G}f(Z(t)))$$

$$\frac{\mathrm{d}}{\mathrm{d}t}\ln \mathbb{E}(Z(t{)}^{\ell })\le \ell (1-b)+\gamma \frac{{\mathrm{e}}^{(1-b)(\ell -1)t}}{\mathbb{E}(Z(t{)}^{\ell })}.$$

Full text

Keywords: Yule-Simon model, preferential attachment, memory, continuous state branching process, Crump-Mode-Jagers branching process, heavy tail distributions.

AMS subject classifications: 60J85 ; 05C85.

1 Introduction

Partly inspired by the earlier work of G. U. Yule, Herbert A. Simon [16] argued in 1955 that an elementary stochastic model could explain the occurrence of power tail distributions in a variety of empirical data. In short, he introduced a simple random algorithm to produce a long string of words $w_{1}\ldots w_{n}$. This algorithm depends on a parameter $p\in(0,1)$ which, in some sense, measures the innovation, and can be described as follows. Once the first word $w_{1}$ has been written, for each $k=1,\ldots,n-1$, with probability $p$, $w_{k+1}$ is a new word different from all the preceding, and with complementary probability $\bar{p}=1-p$, $w_{k+1}$ is copied from a uniform sample from $w_{1},\ldots,w_{k}$.

Simon’s model can be viewed the germ of many network growth dynamics with preferential attachment that flourished since the turn of the millennium. We merely refer here to [2] for an application to the World Wide Web, and to the textbooks [5, 6, 18] and their bibliographies which contain a wealth of relevant references. During the last decade or so, several works in this field have aimed at taking also into account further relevant features for specific models. In particular, an important issue e.g. in citation networks, is to incorporate aging, or decay of relevance, or fading memory phenomena. Dorogovtsev and Mendes [4, 5] first generalized the Barabási-Albert model by letting nodes loosing attraction as their ages increase. Cattuto et al. [3] modified Simon’s model by assigning different weights to different words; we shall present their framework explicitly later on as it constitutes the basis of the present work. In a somewhat different direction, the linear rate birth processes related to Yule’s model or to classical preferential attachment dynamics, have been replaced by nonlinear time fractional birth processes to emulate the effect of a slowly-decaying memory in [11, 13, 14], whereas [7] rather introduces fitness and age-dependency for those birth processes. Further generalizations can be found in the literature, also refer for instance to [17, 19].

The modification Simon’s model that Cattuto et al. considered concerns the copy mechanism, which is no longer made uniformly at random but rather depends on weights assigned to each word. In general, the weight of a word $w_{i}$ is a function $q(i,k)$ of both the rank $i$ of that word and the current total length $k$ of the string. Recall that the word $w_{k+1}$ is attached at the $k$-th step of the algorithm to the string $w_{1}\ldots w_{k}$; just as for the original model of Simon, $w_{k+1}$ is a new word different from all the preceding with a fixed probability $p$. However , with probability $\bar{p}=1-p$, $w_{k+1}$ is a copy of one of the preceding words sampled now at random with probability proportional to its weight, meaning $w_{k+1}=w_{J}$ for some random index $J$ with distribution

[TABLE]

We refer henceforth to this modified algorithm as the $q$-weighted Simon’s model. When weights $i\mapsto q(i,k)$ increase with the index $i\leq k$, recent words are more likely to be repeated than old ones, henceforth emulating a fading memory effect; Simon’s original model is recovered plainly when all weights are equal. Obviously, assigning weights to words also changes profoundly the mathematical analysis of the algorithm. Notably, the process which counts the number of occurrences of some given world as a function of the number of steps of the algorithm is a Markov chain with explicit transition distributions for Simon’s original model, but the Markov property is lost when weights are not all equal.

Cattuto et al. [3] considered hyperbolic weights, viz. $q(i,k)=1/(\tau+k-i)$, where $\tau>0$ is a characteristic time-scale. Quite recently, [15] dealt with short-ranged memory with $q(i,k)=\mathbbm{1}_{\{k-i\leq\kappa\}}$, where $\kappa>0$ should be thought of as the memory range. In the present work, we shall focus on weights depending as a power function on the rank of the word only,

[TABLE]

where $\alpha>0$ is another parameter. Note that for any $a\in(0,1)$, the probability of repeating some word $w_{i}$ with $i\leq ak$ at the $k$-th step tends to $a^{\alpha+1}$ as $k\to\infty$ for the choice (1), whereas this probability would converge [math] in Cattuto’s setting. Hence, memory fades away more slowly in the present framework than in Cattuto’s.

Our purpose here is not to increase marginally the already rich variety of models that exist in this area, but rather to point out that, despite of the loss of the Markov property, a fairly detailed analysis can be made thanks to connections with some rather simple branching processes that may be interesting in their own right. A priori, the appearance of branching processes in this setting should certainly not come as a surprise, as it is well-know that they play a fundamental role in preferential attachment dynamics. Nonetheless the connection here is somehow less direct, it only holds asymptotically, and it does not seem straightforward to describe a rigorous construction of weighted Simon’s models from branching processes.

To start with, recall two fundamental features of Simon’s original model. First, the proportion of distinct words which are repeated some fixed amount of times converges as the length of the string goes to infinity. Specifically, note that the number of different words which have been used when the string has total length $n$ is close to $np$ when $n\gg 1$. If for every $j\geq 1$, we write $\nu_{n}(j)$ for the number of different words which occur exactly $j$ times in a string of total length $n$, then

[TABLE]

where $\rho=1/\bar{p}$ and ${\mathrm{B}}$ stands for the Beta function. The right-hand side above is a probability mass function called the Yule-Simon distribution (with parameter $\rho$). Second, since ${\mathrm{B}}(j,\rho+1)\sim\Gamma(\rho+1)j^{-(\rho+1)}$ as $j\to\infty$; the tail of the Yule-Simon distribution decays as a power function with exponent $-\rho$.

We now state a simple version of our main result for the $q$-weighted Simon’s model when weights are given by (1). In short, as the length of the string tends to infinity, the proportion of different words with any given occurrence number converges towards some mass distribution $\varphi$. Further, $p_{c}=1-\alpha/(1+\alpha)$ is the critical innovation parameter, in the sense that $\varphi$ has a power tail when $p<p_{c}$, and and exponential tail when $p>p_{c}$.

Theorem 1**.**

Let $\nu_{n}(\ell)$ denote the number of different words which are repeated exactly $\ell$ times in a string of total length $n$ generated by the $q$-weighted Simon’s model, with $q$ given by (1). Then we have:

1. (i)

for every $\ell\geq 1$,

[TABLE]

where $\varphi$ is the probability mass function of some random variable with values in $\mathbb{N}=\{1,2,\ldots\}$, 2. (ii)

if $\bar{p}=1-p>\alpha/(1+\alpha)$, then

[TABLE]

for some $0<C<\infty$, 3. (iii)

if $\bar{p}<\alpha/(1+\alpha)$, then there exists $c>0$ with

[TABLE]

More precisely, we shall describe the probability mass function $\varphi$ that appears in Theorem 1 in two different ways. First, it can be seen as the law of the number of birth events in some continuous state branching process that occurred before an independent exponential random time. Equivalently, it can also be seen as the law of a general (Crump-Mode-Jagers) branching process also evaluated at an independent exponential random time. Although we have not been able to provide any explicit expressions $\varphi$, we shall establish the power or exponential tail behaviors stated in Theorem 1 by making use of classical martingales associated to branching processes.

The rest of this work is organized as follows. Section 2 introduces the notion of attraction of a given word as a function of the length of the string. This enables us to circumvent the lack of the Markov property of the occurrence counting alone by viewing it as a component of a Markovian pair, and then to analyze its asymptotic behavior. Section 3 describe representations of the limiting processes of Section 2 in terms of simple branching processes, and describe their behaviors as time goes to infinity using classical martingales in this setting. Finally, Theorem 1 is proved in Section 4.

2 Occurrence counting and attraction

As it has already been mentioned in the Introduction, an obvious obstacle in the analysis of frequencies of words for a weighted Simon’s model is that the process counting the number of occurrences of a word is not Markov. We shall circumvent this difficulty by considering another natural functional of the algorithm, which enjoys the Markov property and from which occurrence counting can be recovered.

To start with, we say that the $j$-th word in a string is new if $w_{j}\neq w_{i}$ for all $1\leq i<j$ (for any $j\geq 2$, this event has probability $p$); otherwise, we say that the $j$-th word is a repetition. When the $j$-th word is new, we define its occurrence counting and its attraction as a function of length of the string, respectively by

[TABLE]

and

[TABLE]

Observe that the second quantity is proportional to the probability that the $j$-th word will be repeated at the $n$-th step (the normalization is chosen for the purpose of convenience as it should become clear later on). When the $j$-th word is a repetition, we set $N_{j}(n)=A_{j}(n)=0$.

When the $j$-th word is new, the evolution of its occurrence counting and of its attraction bear obvious similarities, especially when $n\gg 1$: if $w_{j}$ is repeated at the $n$-th step, the occurrence counting increases exactly by $1$ and the attraction by slightly less than $1$, and otherwise the occurrence counting is unchanged whereas the attraction slightly decreases. In particular there is the identity

[TABLE]

Our purpose here is to check that up-to a simple time-rescaling, occurrence counting and attraction converge jointly in distribution as the total length of the string goes to $\infty$. To describe their limits, we introduce a time-inhomogeneous Markov process with càdlàg paths in $[0,\infty)$, $A=(A(t))_{t\geq 0}$. The law $P_{u}$ of $A$ depends on a parameter $u\in(0,1)$ which should be thought of as a birth-time. Specifically, under $P_{u}$, we have that $A(t)=0$ for $t<u$ and $A(u)=1$, the (right) slope of the path at time $t>0$ is $-\alpha A(t)/t$, and further for $t\geq u$, jumps of size $1$ occur with intensity $\bar{p}(\alpha+1)A(t)/t$. In other words, its infinitesimal generator at time $t\geq u$ is given by

[TABLE]

for any $a>0$ and any bounded and continuously differentiable function $f:(0,\infty)\to{\mathbb{R}}$. The slope and jump rates being bounded, the existence and uniqueness of this process are immediate. We further denote the counting process of the jumps of $A$ by

[TABLE]

We shall now check that the distribution of the pair of processes $(A,N)=(A(t),N(t))_{t\geq 0}$ under $P_{u}$ arises as the weak limit of a time-rescaled version of the attraction and occurence counting.

Lemma 1**.**

1. (i)

Let $(j(n))_{n\geq 1}$ be a sequence with $j(n)/n\to u\in(0,1)$. The conditional distribution of the pair of time-rescaled processes

[TABLE]

given that the $j(n)$-th word is new converges in the sense of Skorohod towards the law of $(A,N)$ under $P_{u}$. 2. (ii)

Let $(k(n))_{n\geq 1}$ be a second sequence with $k(n)/n\to v\in(0,1)$ and $j(n)\neq k(n)$ for all $n$ sufficiently large. The joint conditional distribution of the two pairs of time-rescaled counting processes

[TABLE]

given that the $j(n)$-th and the $k(n)$-th words are both new converges in the sense of Skorohod towards the law of two pairs of independent processes distributed as $(A,N)$ under $P_{u}$ and under $P_{v}$, respectively.

Proof.

We shall only prove the first assertion, the argument for the second is similar but with heavier notation and details are left to scrupulous readers.

We see from from the dynamics of the $q$-weighted Simon’s model that, given that the $j$-th word is new, the pair $((A_{j}(\ell),N_{j}(\ell)):\ell\geq j)$ is an inhomogeneous Markov chain started at time $j$ from $(1,1)$, with probability transitions given for any $\ell\geq j$, $a>0$ and $k\geq 1$ by

[TABLE]

and

[TABLE]

where

[TABLE]

Then take $j=j(n)$ and $\ell=\ell(n)\sim tn$. It follows that for every bounded function $f:(0,\infty)\times\mathbb{N}\to{\mathbb{R}}$ which is continuously differentiable in the first variable,

[TABLE]

On the other hand, the process $(A,N)$ under $\mathbb{P}_{u}$ is an inhomogeneous Feller process, with $(A(t),N(t))=(0,0)$ for all $t<u$, $(A(u),N(u))=(1,1)$, and thanks to (2), its infinitesimal generator given by the right-hand side above. Our claim can now be derived from basic Markov chains approximation; see e.g. Theorem 19.28 in [9]. ∎

3 Connection to continuous state and Crump-Mode-Jagers branching processes

The time-inhomogeneity of the Markov process $A$ introduced in Section 2 is essentially artificial, in the sense that $A$ actually results from a time-homogeneous Markov process by a deterministic logarithmic time substitution, as we shall now explain. Consider a time homogeneous Markov process $Z=(Z(t))_{t\geq 0}$ on $(0,\infty)$ with infinitesimal generator given for any smooth function $f:(0,\infty)\to{\mathbb{R}}$ by

[TABLE]

where $b\geq 0$ is some parameter. We shall always deal with the situation when the process starts from $Z(0)=1$, and shall not mention the initial state any further. The following observation is immediate by comparing (3) with (2).

Lemma 2**.**

Take

[TABLE]

For any fixed $u\in(0,1)$, the process given by

[TABLE]

has the law $P_{u}$.

The process $Z$ is a simple instance of a continuous state branching process (in short, CSBP). We now recall some basic features in this setting and refer to Section VI.6 in [1] or Chapter 12 in [10] for background. Lamperti described a construction of general CSBPs from Lévy processes with no negative jumps (see Theorem 12.2 in [10]), which we specialize to our setting.

Consider first a standard Poisson process $(\eta_{t})_{t\geq 0}$ started from $\eta_{0}=1$, and then

[TABLE]

So $(\xi_{t})_{t\geq 0}$ is a Lévy process with no negative jumps started from $\xi_{0}=1$, and more precisely, all its jumps have unit size. Next write

[TABLE]

for the first hitting time [math], with the usual convention $\inf\varnothing=\infty$. The indefinite integral $t\mapsto\int_{0}^{t}\mathrm{d}s/\xi_{s}$ yields a bijection from $[0,\zeta)$ to $[0,\infty)$; we denote the inverse bijection by $T(\cdot)$. Then the time-changed process $(\xi_{T(t)})_{t\geq 0}$ is a version of $Z$.

Conversely, one notes the identity

[TABLE]

so that if we define

[TABLE]

then we can identify $B(t)=\eta_{T(t)}$. That is, $B(t)$ is the number of birth events (i.e. the number of jumps of $Z$) occurring on the time-interval $[0,t]$, agreeing that $t=0$ is viewed as the first birth event (i.e. jump time of $Z$). We now derive from Lemma 2 the following:

Corollary 1**.**

Let $b$ be given by (4) and fix any $u\in(0,1)$. In the notation above, the process defined by

[TABLE]

has the same law as $N=(N(t))_{t\geq 0}$ under $P_{u}$.

As we shall see later on, the tail estimates in Theorem 1(ii-iii) depend crucially on the behavior of $N(1)$ under $P_{u}$ as $u\to 0+$. Corollary 1 enables us to translate this question in terms of the asymptotic behavior of $B(t)$ as $t\to\infty$. In this direction, it is well-known that, loosely speaking, the large time asymptotic behavior of a supercritical branching process depends on integrability properties of a remarkable martingale. In the present setting, the CSBP $Z$ is supercritical if and only if $b<1$, and we claim the following.

Lemma 3**.**

When $b<1$, the process

[TABLE]

is a martingale which is bounded in $L^{k}(\mathbb{P})$ for all $k\geq 1$.

Proof.

That $W$ is a martingale is well-known and indeed immediate from (3). We shall now check by induction that for every $k\geq 1$, there exists some constant $c_{k}<\infty$ such that

[TABLE]

For $k=1$, (8) is actually an equality with $c_{1}=1$, thanks to the martingale property of $W$. Let us now assume that for some $\ell\geq 2$, (8) holds for all $k=1,\ldots,\ell-1$. Take $f(x)=x^{\ell}$, so that

[TABLE]

Combining Kolmogorov’s forward equation

[TABLE]

with (8), we deduce that for some $\gamma>0$, there is the inequality

[TABLE]

On the other hand, we know from Jensen’s inequality that

[TABLE]

and hence

[TABLE]

We conclude that

[TABLE]

So (8) also holds for $k=\ell$ and the proof is complete. ∎

We write $W_{\infty}$ for the terminal value of the martingale $W$ in Lemma 3 and $W^{*}=\sup_{t\geq 0}W_{t}$ for its overall supremum. We immediately deduce the following strong limit theorem for the number of birth events $B$.

Corollary 2**.**

When $b<1$, there is the convergence

[TABLE]

Further,

[TABLE]

Proof.

Writing

[TABLE]

yields the first claim by dominated convergence. The inequality of the second claim is obvious and the assertion that $W^{*}\in L^{k}(\mathbb{P})$ follows from Lemma 3 and Doob’s inequality. ∎

Remark 1**.**

In the case $b=0$, $Z$ is the standard Yule process, and for each $t$, $Z(t)$ has thus the geometric distribution with parameter ${\mathrm{e}}^{-t}$. Theoretically, the calculation in the proof of Lemma 3 allows us to compute inductively the entire moments of $Z(t)$ for any $b>0$, and then also those of $B(t)$ by standard methods. Because $Z$ is stochastically dominated by a Yule process, the moment problem is determinate. So in theory, this approach enables to characterize the law of $B(t)$; however I have not been able to get an explicit formula.

The final lemma of this section deals with the sub-critical case, and will provide the key to Theorem 1(iii).

Lemma 4**.**

When $b>1$, $Z$ is subcritical, $\lim_{t\to\infty}Z_{t}=0$ a.s. and one has

[TABLE]

for $c=\ln b+b^{-1}-1>0$.

Proof.

The first two assertions should be plain, and we just need to establish the displayed identity. To start with, we recall from Lamperti’s transformation that the integral $\int_{0}^{\infty}Z(s)\mathrm{d}s$ coincides with the hitting time $\zeta$ of [math] by the Lévy process $\eta$.

The Laplace exponent $\psi$ of the Lévy process $\xi$ is $\psi(\lambda)=b\lambda-1+{\mathrm{e}}^{-\lambda}$ for $\lambda\in{\mathbb{R}}$, meaning that ${\mathbb{E}}(\exp(-\lambda(\xi_{t}-1)))=\exp(t\psi(\lambda))$. This function reaches its minimum at $-\ln b<0$, with $\min\psi=\psi(-\ln b)=-b\ln b-1+b<0$. It follows from a well-known formula for the Laplace transform of first-passage times (see e.g. Theorem 3.12 in [10]) that

[TABLE]

which establishes our claim. ∎

We now conclude this section by pointing at a connection with another branching process, namely of Crump-Mode-Jagers (in short, C-M-J) type. Recall that a C-M-J branching process is a model for the evolution of a population in continuous time, where individuals beget children according to independent copies of a point process $\Xi$ on $(0,\infty)$ and locations of atoms are interpreted as birth of children. In other words, a C-M-J process can be constructed from a branching random walk on $[0,\infty)$ with reproduction law given by the distribution of $\Xi$, by viewing any atom, say located at $s$, at any generation of the branching random walk, as an individual born at time $s$ in the C-M-J process. Then individuals alive at time $t$ in the C-M-J process correspond to locations $s\leq t$ of atoms in the branching random walk.

Now take for $\Xi$ a Poisson point measure with intensity ${\mathrm{e}}^{-bt}\mathrm{d}t$, and assign to each individual in the C-M-J process a size which decays exponentially with time. Specifically the size of an individual at age $a$ is ${\mathrm{e}}^{-ab}$, so that each individual in the C-M-J process begets children at rate given precisely by its current size. It follows that, if we further assume that the C-M-J process starts from a single ancestor at time [math], then the process describing the sum of sizes of individuals as a function of time, is a version of the CSBP $Z$. As a consequence, the process $(B(t))_{t\geq 0}$ has the same law as the process of the number of individuals alive of the population in this C-M-J process. In this framework, the so-called Malthus exponent is readily identified with $1-b$, and the martingale $W$ is known as the intrinsic martingale. We refer to [12, 8] which provide in particular a criterion for uniformly integrability of intrinsic martingales and demonstrate the importance of their roles in limit processes for C-M-J branching processes. Notably, the first part of Corollary 2 can also be derived from Theorem 5.4 in [12].

4 Proof of Theorem 1

We have now all the ingredients needed for the proof of Theorem 1. To start with, recall that for every $\ell\geq 1$,

[TABLE]

denotes the number of different words which have been repeated exactly $\ell$ times when the string reaches the length $n$. Recall also from Section 2 the definition of the law $P_{u}$ for any $u\in(0,1)$, and of the counting process of jumps $N$. We now define a probability mass function $\varphi$ on $\mathbb{N}$ by

[TABLE]

With these notation at hand, we easily deduce Theorem 1(i) from Lemma 1 by first and second moment calculations,.

Proof of Theorem 1(i).

Let $U$ be a uniform random variable on $(0,1)$ independent of the weighted Simon’s model. For every $n\geq 1$, write $j(n)=\lceil nU\rceil$, so that $j(n)$ is uniformly distributed on $\{1,\ldots,n\}$ and $j(n)/n\to U$. Recall that for every $j\geq 2$, the probability that the $j$-th word is new equals $p$ and that $N_{j}(n)=0$ if the $j$-th word is a repetition. We immediately deduce from Lemma 1(i) that

[TABLE]

Next, let $V$ be a second uniform random variable on $(0,1)$, independent of $U$ and the weighted Simon’s model, and set $k(n)=\lceil nV\rceil$. So $k(n)$ is uniformly distributed on $\{1,\ldots,n\}$ and independent of $j(n)$, and writing

[TABLE]

we deduce similarly from Lemma 1(ii) that

[TABLE]

which establishes our claim. ∎

The proof Theorem 1(ii-iii) relies on the CSBP $Z$ introduced in Section 3; we henceforth let the parameter $b>0$ there be given by (4).

Proof of Theorem 1(ii).

Thanks to Corollary 1, we have

[TABLE]

where $\epsilon$ is a random variable independent of $B$ and with the standard exponential distribution.

Next define for every $t\geq 0$ the variables

[TABLE]

and observe from Corollary 2 that for any $k\geq 1$, we have by dominated convergence

[TABLE]

We first treat the lower-bound. Fix $t>0$. By the definition of $\beta_{*}(t)$, the event $B(\alpha b^{-1}\epsilon)>k$ holds whenever

[TABLE]

and hence a fortiori whenever

[TABLE]

This yields the lower bound

[TABLE]

On the one-hand, recalling that $b$ is given by (4) and that $\epsilon$ is exponentially distributed and independent of $B$, we get

[TABLE]

On the other hand, since $\beta_{*}(t)\leq(1-b)^{-1}W^{*}\in L^{\ell}(\mathbb{P})$ for any $\ell\geq 1$ (see Corollary 2), we have by the Markov’s inequality

[TABLE]

Putting the pieces together, we have shown that for any $t>0$,

[TABLE]

Letting $t\to\infty$ and using (10), we conclude that

[TABLE]

We now turn our attention to the upper-bound and fix $t>0$. By the definition of $\beta^{*}(t)$, there is the inclusion of events

[TABLE]

with

[TABLE]

Using the identity $\Lambda_{1}\cup\Lambda_{2}=\Lambda_{1}\cup(\Lambda_{1}^{\rm c}\cap\Lambda_{2})$ and then observing that

[TABLE]

we see that $B(\alpha b^{-1}\epsilon)>k$ implies

[TABLE]

This yields the upper-bound

[TABLE]

By the same argument as for the lower-bound, we arrive at

[TABLE]

Letting $t\to\infty$ and using (10), we conclude that

[TABLE]

∎

Remark 2**.**

The proof above of Theorem 1(ii) identifies the constant $C$ there as

[TABLE]

In this direction, recall that for $\alpha=0$, one has $b=0$, so $Z$ is simply the Yule process and $W_{\infty}$ follows the standard exponential distribution. This yields $C=\Gamma(1+1/\bar{p})$, which agrees with what was known for Simon’s original model.

Finally, the last part of Theorem 1 follows readily from Lemma 4.

Proof of Theorem 1(iii).

We take $c=\ln b+b^{-1}-1>0$ and write $N(\infty)$ for the limit as $t\to\infty$ of the counting process $N$. We have from the definition of $\varphi$ and the inequality $N(1)\leq N(\infty)$ that

[TABLE]

Since $b>1$, we know from Corollary 1 and Lemma 4 that the law of $N(\infty)$ under $P_{u}$ is the same as that of $b\int_{0}^{\infty}Z_{s}\mathrm{d}s$. Lemma 4 further shows that $E_{u}({\mathrm{e}}^{cN(\infty)})=b$ for any $u\in(0,1)$, and we conclude that

[TABLE]

∎

To conclude this work, we observe that the characterization (9) of the limiting mass probability function $\varphi$ in terms of a branching process extends the fact that the Yule-Simon distribution can be realized as the law of a standard Yule process evaluated at an independent exponentially distributed random time with parameter $\rho=1/\bar{p}$. It is also interesting to stress that similar variables -but associated to different branching processes- also arise in [7], even though the framework of [7] and the present one seem to be rather different (in particular, the degree distribution plays a key role in there, whereas we here are rather concerned with the distribution of the total population size).