Dependence between Path-length and Size in Random Digital Trees

TL;DR

This paper investigates the dependence between size and path length in random digital trees, revealing asymptotic independence in asymmetric cases and strong dependence with fluctuations in symmetric cases, contrasting prior results.

Contribution

It uncovers novel dependence behaviors in digital trees, showing independence or dependence based on symmetry, and extends findings to other digital tree classes.

Findings

Asymptotic independence in asymmetric digital tries.

Strong dependence with periodic fluctuations in symmetric tries.

Different correlation behaviors in various digital tree classes.

Abstract

We study the size and the external path length of random tries and show that they are asymptotically independent in the asymmetric case but strongly dependent with small periodic fluctuations in the symmetric case. Such an unexpected behavior is in sharp contrast to the previously known results on random tries that the size is totally positively correlated to the internal path length and that both tend to the same normal limit law. These two dependence examples provide concrete instances of bivariate normal distributions (as limit laws) whose correlation is $0$, $1$ and periodically oscillating. Moreover, the same type of behaviors is also clarified for other classes of digital trees such as bucket digital trees and Patricia tries.

Tables

Table 1. Table 1: Asymptotic patterns of the means and the variances of the shape parameters discussed in this paper. Here ℱ ​ [ ⋅ ] ​ ( n ) ℱ delimited-[] ⋅ 𝑛 \mathscr{F}[\cdot](n) differs from one occurrence to another and h = − p ​ log ⁡ p − q ​ log ⁡ q ℎ 𝑝 𝑝 𝑞 𝑞 h=-p\log p-q\log q denotes the entropy. Expressions for g k ( 1 ) subscript superscript 𝑔 1 𝑘 g^{(1)}_{k} and g k ( 3 ) subscript superscript 𝑔 3 𝑘 g^{(3)}_{k} will be given below. All three random variables S n , N n , K n subscript 𝑆 𝑛 subscript 𝑁 𝑛 subscript 𝐾 𝑛 S_{n},N_{n},K_{n} are asymptotically normally distributed.

Shape parameters	$\frac{1}{n}\left(\text{mean}\right)\sim$	$\frac{1}{n}\left(\text{variance}\right)\sim$
Size $S_n$	$ℱ[\cdot ]\left(n\right)$	$ℱ\left[g^{\left(1\right)}\right]\left(n\right)$
NPL $N_n$	$\frac{𝔼\left(S_n\right)}{n}\cdot \frac{\log n}{h}$	$\frac{𝕍\left(S_n\right)}{n}\cdot \frac{{\left(\log n\right)}^2}{h^2}$
KPL $K_n$	$\frac{\log n}{h}+ℱ[\cdot ]\left(n\right)$	$\frac{pq{\log }^2\frac{p}{q}}{h^2}$ $\cdot \frac{\log n}{h}+ℱ\left[g^{\left(3\right)}\right]\left(n\right)$
Depth $D_n$	$𝔼\left(D_n\right)=\frac{𝔼\left(K_n\right)}{n}$	$𝕍\left(D_n\right)=\frac{𝕍\left(K_n\right)}{n}+O\left(1\right)$

Table 2. Table 2: A comparison of the correlation coefficients defined on random tries and on random m 𝑚 m -ary search trees: the size of m 𝑚 m -ary search trees corresponds to the space requirement, and the KPL and NPL are defined similarly as in tries.

trees	$\rho (S_n,K_n)$	$\rho (S_n,N_n)$
tries	$\{\begin{array}{c}p\ne q:\to 0\\ p=q:\text{periodic}\end{array}$	$\sim 1$
$m$-ary search trees	$\{\begin{array}{c}3⩽m⩽26:\to 0\\ m⩾27:\text{periodic}\end{array}$

Table 3. Table 3: Numerical approximations to the average values of the periodic functions F ​ ( n ) 𝐹 𝑛 F(n) arising in the asymptotic estimate ρ ​ ( S n , K n ) ∼ F ​ ( n ) similar-to 𝜌 subscript 𝑆 𝑛 subscript 𝐾 𝑛 𝐹 𝑛 \rho(S_{n},K_{n})\sim F(n) for the symmetric case and m = 2 , … , 6 𝑚 2 … 6 m=2,\dots,6 .

$m$	$2$	$3$	$4$	$5$	$6$
$\begin{array}{c}\text{average value of the}\\ \text{periodic fluctuations}\end{array}$	$0.927$	$0.925$	$0.924$	$0.922$	$0.921$

Table 4. Table 4: Numerical approximations to the average values of the periodic functions F ​ ( n ) 𝐹 𝑛 F(n) arising in the asymptotic estimate ρ ​ ( S n , K n ) ∼ F ​ ( n ) similar-to 𝜌 subscript 𝑆 𝑛 subscript 𝐾 𝑛 𝐹 𝑛 \rho(S_{n},K_{n})\sim F(n) for the symmetric case and m = 3 , … , 6 𝑚 3 … 6 m=3,\dots,6 .

$m$	$3$	$4$	$5$	$6$
$\begin{array}{c}\text{average value of the}\\ \text{periodic fluctuations}\end{array}$	$0.751$	$0.814$	$0.841$	$0.856$

Equations

$$\{\begin{array}{rl}{S}_n& \stackrel{d}{=}{S}_{B_n}+S_{n-B_n}^{\ast }+1,\\ K_n& \stackrel{d}{=}{K}_{B_n}+K_{n-B_n}^{\ast }+n,\\ N_n& \stackrel{d}{=}{N}_{B_n}+N_{n-B_n}^{\ast }+S_{B_n}+S_{n-B_n}^{\ast },\end{array}(n⩾2),$$

$$\mathcal{F}[g](z)=\{\begin{array}{ll}{\sum }_{k\in \mathbb{Z}}{g}_k{z}^{-{\chi }_k},& \text{if }\frac{\log p}{\log q}\in \mathbb{Q};\\ g_0,& \text{if }\frac{\log p}{\log q}\notin \mathbb{Q},\end{array}$$

$$\lambda ={\lambda }_p:=\frac{pq{\log }^2\frac{p}{q}}{h^3}=\frac{(p{\log }^{2}p+q{\log }^{2}q)-h^2}{h^3}$$

$$\rho (S_n,N_n)\sim 1(0<p<1).$$

$$\left(\frac{S_n-\mathbb{E}(S_n)}{\sqrt{\mathbb{V}(S_n)}},\frac{N_n-\mathbb{E}(N_n)}{\sqrt{\mathbb{V}(N_n)}}\right)\stackrel{d}{⟶}({\mathcal{N}}_1,{\mathcal{N}}_1),$$

$${\left(\frac{S_n-\mathbb{E}(S_n)}{\sqrt{\mathbb{V}(S_n)}},\frac{N_n-\mathbb{E}(N_n)}{\sqrt{\mathbb{V}(N_n)}}\right)}^{⊺}\stackrel{d}{⟶}{\mathcal{N}}_2(0,E_2),$$

$$\mathrm{Cov}(S_n,K_n)\sim n\mathcal{F}[g^{(2)}](n),$$

$$\rho (S_n,K_n)\sim \{\begin{array}{ll}0,& \text{if}p\ne \frac{1}{2}\\ F(n),& \text{if}p=\frac{1}{2}.\end{array}$$

$${\left(\frac{S_n-\mathbb{E}(S_n)}{\sqrt{\mathbb{V}(S_n)}},\frac{K_n-\mathbb{E}(K_n)}{\sqrt{\mathbb{V}(K_n)}}\right)}^{⊺}\stackrel{d}{⟶}{\mathcal{N}}_2(0,I_2),$$

$${\Sigma }_n^{-\frac{1}{2}}\left(\begin{array}{c}{S}_n-\mathbb{E}(S_n)\\ K_n-\mathbb{E}(K_n)\end{array}\right)\stackrel{d}{⟶}{\mathcal{N}}_2(0,I_2),$$

$${\Sigma }_n:=n\left(\begin{array}{cc}\mathcal{F}[g^{(1)}](n)& \mathcal{F}[g^{(2)}](n)\\ \mathcal{F}[g^{(2)}](n)& \mathcal{F}[g^{(3)}](n)\end{array}\right).$$

$${\Sigma }_n:=n\left(\begin{array}{cc}\mathcal{F}[g^{(1)}](n)& \mathcal{F}[g^{(2)}](n)\\ \mathcal{F}[g^{(2)}](n)& \lambda \log n+\mathcal{F}[g^{(3)}](n)\end{array}\right).$$

$${\Sigma }_n^{-\frac{1}{2}}\left(\begin{array}{c}{S}_n-\mathbb{E}(S_n)\\ K_n-\mathbb{E}(K_n)\end{array}\right)\stackrel{d}{⟶}{\mathcal{N}}_2(0,I_2).$$

$${\tilde{f}}_{1,0}(z):=e^{-z}\sum _{n⩾0}\mathbb{E}(S_n)\frac{z^n}{n!},\text{and}{\tilde{f}}_{0,1}(z):=e^{-z}\sum _{n⩾0}\mathbb{E}(K_n)\frac{z^n}{n!}.$$

$$\{\begin{array}{rl}{\tilde{f}}_{1,0}(z)& ={\tilde{f}}_{1,0}(pz)+{\tilde{f}}_{1,0}(qz)+1-(1+z)e^{-z},\\ {\tilde{f}}_{0,1}(z)& ={\tilde{f}}_{0,1}(pz)+{\tilde{f}}_{0,1}(qz)+z(1-e^{-z}).\end{array}$$

$${\tilde{f}}_{1,0}(z)\sim z\mathcal{F}[\cdot ](z),\text{and}{\tilde{f}}_{0,1}(z)\sim h^{-1}z\log z+z\mathcal{F}[\cdot ](z),$$

$${\tilde{f}}_{2,0}(z):=e^{-z}\sum _{n⩾0}\mathbb{E}(S_n^2)\frac{z^n}{n!},\text{and}{\tilde{f}}_{0,2}(z):=e^{-z}\sum _{n⩾0}\mathbb{E}(K_n^2)\frac{z^n}{n!},$$

$$\{\begin{array}{rl}{\tilde{V}}_S(z)& :={\tilde{f}}_{2,0}(z)-{\tilde{f}}_{1,0}(z{)}^2-z{\tilde{f}}_{1,0}^{\prime }(z{)}^2,\\ {\tilde{V}}_K(z)& :={\tilde{f}}_{0,2}(z)-{\tilde{f}}_{0,1}(z{)}^2-z{\tilde{f}}_{0,1}^{\prime }(z{)}^2,\end{array}$$

$${\tilde{f}}_{1,1}(z):=e^{-z}\sum _{n⩾0}\mathbb{E}(S_n{K}_n)\frac{z^n}{n!},$$

$${\tilde{f}}_{1,1}(z)$$

$$+pz{\tilde{f}}_{1,0}^{\prime }(pz)+qz{\tilde{f}}_{1,0}^{\prime }(qz)+{\tilde{f}}_{0,1}(pz)+{\tilde{f}}_{0,1}(qz)+z(1-e^{-z}).$$

$$\tilde{C}(z)={\tilde{f}}_{1,1}(z)-{\tilde{f}}_{1,0}(z){\tilde{f}}_{0,1}(z)-z{\tilde{f}}_{1,0}^{\prime }(z){\tilde{f}}_{0,1}^{\prime }(z),$$

$$\tilde{C}(z)=\tilde{C}(pz)+\tilde{C}(qz)+{\tilde{h}}_1(z)+{\tilde{h}}_2(z),$$

$${\tilde{h}}_1(z)=pqz\left({\tilde{f}}_{1,0}^{\prime }(pz)-{\tilde{f}}_{1,0}^{\prime }(qz)\right)\left({\tilde{f}}_{0,1}^{\prime }(pz)-{\tilde{f}}_{0,1}^{\prime }(qz)\right),$$

$$\begin{array}{rl}{\tilde{h}}_2(z)& =z{e}^{-z}\left({\tilde{f}}_{1,0}(pz)+{\tilde{f}}_{1,0}(qz)+p(1-z){\tilde{f}}_{1,0}^{\prime }(pz)+q(1-z){\tilde{f}}_{1,0}^{\prime }(qz)\right)\\ & +e^{-z}\left((1+z){\tilde{f}}_{0,1}(pz)+(1+z){\tilde{f}}_{0,1}(qz)-p{z}^2{\tilde{f}}_{0,1}^{\prime }(pz)-q{z}^2{\tilde{f}}_{0,1}^{\prime }(qz)\right)\\ & +z{e}^{-z}\left(1-(1+z^2)e^{-z}\right).\end{array}$$

$$\tilde{C}(z)=\frac{1}{2\pi i}{\int }_{-\frac{3}{2}-i\infty }^{-\frac{3}{2}+i\infty }\frac{\mathcal{M}[{\tilde{h}}_1(z)+{\tilde{h}}_2(z);s]}{1-p^{-s}-q^{-s}}{z}^{-s}\mathrm{d}s,$$

$$\tilde{C}(z)\sim z\mathcal{F}[g^{(2)}](z),$$

$$\mathrm{Cov}(S_n,K_n)\sim n\mathcal{F}[g^{(2)}](n).$$

$$\begin{array}{rl}{g}_k^{(2)}& =\frac{\Gamma ({\chi }_k)}{h}\left(1-\frac{{\chi }_k+2}{2^{{\chi }_k+1}}\right)-\frac{1}{h^2}\sum _{j\in \mathbb{Z}\setminus \{0\}}\Gamma ({\chi }_{k-j}+1)({\chi }_j-1)\Gamma ({\chi }_j)\\ & -\frac{\Gamma ({\chi }_k+1)}{h^2}\left(\gamma +1+\psi ({\chi }_k+1)-\frac{p{\log }^{2}p+q{\log }^{2}q}{2h}\right)\\ & +\frac{1}{h}\sum _{\ell ⩾2}\frac{(-1{)}^{\ell }(p^{\ell }+q^{\ell })}{\ell !(1-p^{\ell }-q^{\ell })}\Gamma ({\chi }_k+\ell -1)(2{\ell }^2-2\ell +1+{\chi }_k(2\ell -1)),\end{array}$$

$$g_k^{(2)}$$

$$\displaystyle g^{(1)}_{k}=-\frac{\Gamma(\chi_{k}-1)\chi_{k}(\chi_{k}+1)^{2}}{4\log 2}+\frac{2}{\log 2}\sum_{\ell\geqslant 1}\frac{(-1)^{\ell}\Gamma(\chi_{k}+\ell)\ell\bigl{(}\ell(\chi_{k}+\ell)-1\bigr{)}}{(\ell+1)!(2^{\ell}-1)}.$$

Full text

AMS 2010 Subject Classifications. 60C05 60F05 68P05 05C05 68W40

Keywords. Random tries, covariance, total path length, Pearson’s correlation coefficient, asymptotic normality, poissonization/de-Poissonization, integral transform, contraction method.

1 Introduction

Tries are one of the most fundamental tree-type data structures in computer algorithms; see Knuth [18] and Mahmoud [19] for a general introduction. Their general efficiency depends on several shape parameters, the principal ones including the depth, the height, the size, the internal path-length (IPL), and the external path-length (EPL); see below for a more precise description of those studied in this paper. While most of these measures have been extensively investigated in the literature, we are concerned here with the question: how does the EPL depend on the size in a random trie? Surprisingly, while the pair $(\text{IPL},\text{size})$ is known to have asymptotic correlation coefficient tending to one and to have the same normal limit law after each being properly normalized (see [10, 12]), this paper aims to show that the pair $(\text{EPL},\text{size})$ exhibits a completely different behavior depending on the parameter of the underlying random bits being biased or unbiased. This is a companion paper to [2] where we clarified the dependence structure of another class of search trees in computer algorithms.

Given a sequence of binary strings (or keys), one can construct a binary trie (very similar to constructing a dictionary of binary words) as follows. If $n=1$, then the trie consists of a single root-node holding the sole string; if $n\geqslant 2$, the root is used to direct the strings into the corresponding subtree: if the first bit of the input string is [math] (or $1$), then the string goes to the left (or right) subtree; strings directed to the same subtree are then processed recursively in the same manner but instead of splitting according to the first bit, the second bit of each string is then used. In this way, a binary dictionary-type tree with two types of nodes is constructed: external nodes for storing strings and internal nodes for splitting the strings; see Figure 1 for a trie of seven strings.

The random trie model we consider here assumes that each of the $n$ binary keys is an infinite sequence of independent Bernoulli bits each with success probability $0<p<1$. Then the trie constructed from this sequence is a random trie.

We define three shape parameters in a random trie of $n$ strings:

• •

Size $S_{n}$: the total number of internal nodes used (the circle nodes in Figure 1);

• •

IPL (or node path-length, NPL) $N_{n}$: the sum of the distance between the root and each internal node;

• •

EPL (or key path-length, KPL) $K_{n}$: the sum of the distance between the root and each external node.

We will use mostly NPL in place of IPL, and KPL in place of EPL, the reason being an easier comparison with the corresponding results derived for random $m$-ary search trees in the companion paper [2]; see below for more details.

By the recursive definition and our model assumption, we have the following recurrence relations

[TABLE]

with the initial conditions $S_{n}=K_{n}=N_{n}=0$ for $n\leqslant 1$, where $B_{n}=\text{Binom}(n,p)$ denotes a binomial distribution with parameters $n$ and $p\in(0,1)$. Also $(S_{n}^{*}),(K_{n}^{*})$, and $(N_{n}^{*})$ are independent copies of $(S_{n}),(K_{n})$ and $(N_{n})$, respectively. While many stochastic properties of these random variables are known (see Clément et al. [3], Devroye [5] and [10] and many references cited there), much less attention has been paid to their correlation and dependence structure.

The asymptotic behaviors of the moments of random variables defined on tries typically depend on the ratio $\frac{\log p}{\log q}$ being rational or irrational, where $q=1-p$. So we introduce, similar to [10], the notation

[TABLE]

where $g_{k}$ represents a sequence of (Fourier) coefficients and $\chi_{k}=\frac{2rk\pi i}{\log p}$ when $\frac{\log p}{\log q}=\frac{r}{l}$ with $r$ and $l$ coprime. In simpler words, $\mathscr{F}[g](z)$ is a periodic function in the rational case, and a constant in the irrational case. We also use $\mathscr{F}[\cdot](z)$ as a generic symbol if the exact form of the underlying sequence matters less, and in this case each occurrence may not represent the same function.

With this notation, the asymptotics of the mean and the variance are summarized in the following table; see [10, 15, 19] and the references therein for more information.

Note specially that the leading constant

[TABLE]

in the asymptotic approximation to $\mathbb{V}(K_{n})$ equals zero when $p=q$, implying that $\mathbb{V}(K_{n})$ is not of order $n\log n$ but of linear order in the symmetric case. This change of order can be regarded as the source property distinguishing between the dependence and independence of $K_{n}$ on $S_{n}$.

On the other hand, we have the relation $\mathbb{E}(K_{n})=\mathbb{E}(D_{n})n$ between the external path length and the depth $D_{n}$, which is defined to be the distance between the root and a randomly chosen external node (each with the same probability). Furthermore, we also have the asymptotic equivalent $\mathbb{V}(K_{n})\sim\mathbb{V}(D_{n})n$ when $p\neq 1/2$ (or $\lambda>0$), and a central limit theorem for $D_{n}$; see Devroye [4].

From Table 1, we see roughly that each internal node contributes $\frac{\log n}{h}$ to $N_{n}$, namely, that $N_{n}\approx S_{n}\cdot\frac{\log n}{h}$. Indeed, it was proved in [10] that the correlation coefficient of $S_{n}$ and $N_{n}$ satisfies

[TABLE]

Such a linear correlation was further strengthened in [12], where it was proved that both random variables tend to the same normal limit law $\mathcal{N}_{1}$ (with zero mean and unit variance)

[TABLE]

where $\stackrel{{\scriptstyle d}}{{\longrightarrow}}$ denotes convergence in distribution. In terms of the bivariate normal law $\mathcal{N}_{2}$ (see Tong [27]), we can write

[TABLE]

where $E_{2}=\scriptsize\begin{pmatrix}1&1\\ 1&1\end{pmatrix}$ is a singular matrix and $\mathbf{A}^{\intercal}$ denotes the transpose of matrix $\mathbf{A}$.

We show that the correlation and dependence of $K_{n}$ on $S_{n}$ are drastically different. We start with their correlation coefficient.

Theorem A**.**

The covariance of the number of internal nodes and KPL in a random trie of $n$ strings satisfies

[TABLE]

where $g^{(2)}_{k}$ is given in Proposition A below, and their correlation coefficient satisfies

[TABLE]

Here $F(n)=\frac{\mathscr{F}[g^{(2)}](n)}{\sqrt{\mathscr{F}[g^{(1)}](n)\mathscr{F}[g^{(3)}](n)}}$ is a periodic function with average value $0.927\cdots$.

The result (4) is to be compared with (3) (which holds for all $p\in(0,1)$): the surprising difference here comes not only from the (common) distinction between $p=\frac{1}{2}$ and $p\neq\frac{1}{2}$ but also from the (less expected) intrinsic asymptotic nature.

Furthermore, we show that this different behavior cannot be ascribed to the weak measurability of nonlinear dependence of Pearson’s correlation coefficient because the limiting distribution also exhibits a similar dependence pattern. (For the univariate central limit theorems implied by the result below, see Jacquet and Régnier [14] where such results were first established.)

Theorem B**.**

• (i)

For $p\neq\frac{1}{2}$, we have

[TABLE]

where $I_{2}$ denotes the $2\times 2$ identity matrix.

• (ii)

For $p=\frac{1}{2}$, we have

[TABLE]

where $\Sigma_{n}$ denotes the (asymptotic) covariance matrix of $S_{n}$ and $K_{n}$:

[TABLE]

Alternatively, we may define

[TABLE]

Then both cases can be stated in one as

[TABLE]

On the other hand, since for bivariate normal distribution, zero correlation implies independence (see [27]), it is more transparent to split the statement into two cases. See Figure 3 for (Monte Carlo) 3D-plots of the joint distributions of $(S_{n},K_{n})$ when $n=10^{7}$.

These results are to be compared with the corresponding ones for random $m$-ary search trees [2], and the differences for correlation coefficients are summarized in Table 2.

Furthermore, the joint distribution for $m$-ary search trees undergoes a phase change at $m=26$: if the branching factor $m$ satisfies $3\leqslant m\leqslant 26$, then the space requirement is asymptotically independent with the KPL and NPL, while for $m\geqslant 27$, their limiting joint distributions contain periodic fluctuations and are dependent; see [2] for more information.

The dependence phenomena as those discovered in this paper are not limited to random tries and have indeed a wider range of connections. They also appear in different forms in other structures and algorithms with an underlying binomial splitting process; see Flajolet [6] and [10, 13] for references on data structures, algorithms, conflict resolution protocols and stochastic models. A typical example is the dependence between the number of coin-tossings (or bits generated, or bits inspected) and the number of partitioning rounds in (i) CTM tree algorithm (see Rom and Sidi [25]), (ii) bucket sort (see [18] and Mahmoud et al. [20]), (iii) RS Algorithm for generating random permutations (see Bacher et al. [1]), and (iv) initializing radio networks (see Myoupo et al. [21]). We will also present the results without proof for three other classes of digital trees in the last section.

Our approach is mostly analytic and it is unknown if our results can be characterized by probabilistic arguments. Indeed, we believe that the less expected results we discovered are of special interest to probabilists as more structural interpretation or characterization remains to be clarified.

An extended abstract of this paper appeared in the online proceedings of the 27th International Meeting on Probabilistic, Combinatorial and Asymptotic Methods for the Analysis of Algorithms (Kraków, Poland; July 4–8, 2016); see [9]. More details of the proofs, as well as a section on extensions are added in this version (some of them in an appendix). Also, we corrected the plots of Figure 3 in [9]. The extended abstract [9] was peer-reviewed and we incorporated the comments and suggestions of the referees into this version.

2 Covariance and Correlation Coefficient

In this section, we prove Theorem A on the asymptotics of the covariance and correlation coefficient of $S_{n}$ and $K_{n}$, where we content ourselves with a detailed sketch of the method because similar proofs have been given in [10]. In fact, we will also need the variances of $S_{n}$ and $K_{n}$, whose derivations will be recalled below and which have been known for some time; see Jacquet and Régnier [14], Kirschenhofer and Prodinger [16], Kirschenhofer et al. [17], Régnier and Jacquet [24]) and [10]. See also Table 1 for a brief summary of these results.

Our method of proof is based on the by-now standard two-stage approach relying on the theory of analytic de-Poissonization and Mellin transform whose origin can be traced back to Jacquet and Régnier [14]. See Flajolet et al. [7] for a survey on Mellin transform, and Jacquet and Szpankowski [15] for a survey on analytic de-Poissonization. For the computation of the covariance, the manipulation can be largely simplified by the additional notions of Poissonized variance and admissible functions further developed in our previous papers [10, 13].

The starting point of our analysis is the recurrence satisfied by $S_{n}$ and $K_{n}$ in (1). A standard means in the computation of moments of $S_{n}$ and $K_{n}$ is the Poisson generating function, which corresponds to the moments of $S_{n}$ and $K_{n}$ with $n$ replaced by a Poisson random variable with parameter $z$ (this step is called Poissonization).

More precisely, define the Poisson generating function of $\mathbb{E}(S_{n})$ and that of $\mathbb{E}(K_{n})$:

[TABLE]

Then the recurrences (1) lead to the functional equations

[TABLE]

From these equations, we obtain, by Mellin transform techniques [7],

[TABLE]

for large $|z|$ in the half-plane $\Re(z)\geqslant\varepsilon>0$, where $h$ denotes the entropy of Bernoulli($p$). Then, by Cauchy’s integral representation and analytic de-Poissonization techniques [15], we obtain precise asymptotic approximations to $\mathbb{E}(S_{n})$ and to $\mathbb{E}(K_{n})$; see [10] for more details.

Similarly, for the variances $\mathbb{V}(S_{n})$ and $\mathbb{V}(K_{n})$, we introduce the Poisson generating functions of the second moments:

[TABLE]

which then satisfy, by (1), the same type of functional equations as in (5) but with different non-homogeneous parts. Instead of computing directly asymptotic approximations to the second moments, it proves computational more advantageous to consider the Poissonized variances

[TABLE]

and then following the same Mellin-de-Poissonization approach (as for the means) to derive the first and the third asymptotic estimate in the second column of Table 1; again see [10] for details.

It remains to derive the claimed estimate for the covariance. For that purpose, we introduce the Poisson generating function

[TABLE]

which satisfies, again by (1),

[TABLE]

To compute the covariance, it is beneficial to introduce now the Poissonized covariance (see (7) or [10] for similar details)

[TABLE]

which satisfies

[TABLE]

where

[TABLE]

and

[TABLE]

Note that $\tilde{h}_{1}$ is zero when $p=\frac{1}{2}$. Furthermore, from (6) (which can be differentiated since they hold in a sector $\mathscr{S}=\{z\in\mathbb{C}\ :\ \Re(z)\geq\epsilon,|{\rm Arg}(z)|\leqslant\theta_{0}\}$ with $0<\theta_{0}<\pi/2$ in the complex plane), we obtain that $\tilde{h}_{1}(z)=O(|z|)$ and $\tilde{h}_{2}(z)$ is exponentially small for large $|z|$ in $\Re(z)>0$. Also $\tilde{h}_{1}(z)+\tilde{h}_{2}(z)=O(|z|^{2})$ as $z\to 0$. Thus the Mellin transform of $\tilde{h}_{1}(z)+\tilde{h}_{2}(z)$ exists in the strip $\langle-2,-1\rangle$, and we have then the inverse Mellin integral representation

[TABLE]

where $\mathscr{M}[\phi(z);s]:=\int_{0}^{\infty}\phi(z)z^{s-1}\mathrm{d}z$ denotes the Mellin transform of $\phi$; see [7].

Next, again from (6) we see that $\mathscr{M}[\tilde{h}_{1}(z);s]$ can be analytically continued to the vertical line $\Re(s)=-1$ and has no singularities there. Thus, by shifting the line of integration in (10) and computing residues, we obtain

[TABLE]

uniformly for $z$ in a sector.

What is left is the computation of the Fourier coefficients of the periodic function (see Proposition A below). This is in fact the most technical part of the proof because $\tilde{h}_{1}(z)$ contains the product of the two terms $\tilde{f}^{\prime}_{1,0}(pz)-\tilde{f}^{\prime}_{1,0}(qz)$ and $\tilde{f}^{\prime}_{0,1}(pz)-\tilde{f}^{\prime}_{0,1}(qz)$, and thus $\mathscr{M}[\tilde{h}_{1}(z);s]$ is a Mellin convolution integral. In [10], a general procedure was given for the simplification of such integrals (see [10, p. 24 et seq.]). This simplification procedure (see Appendix A for details) and a direct application of the theory of admissible functions of analytic de-Poissonization now yield the following estimate for the covariance of $S_{n}$ and $K_{n}$.

Proposition A**.**

The covariance of $S_{n}$ and $K_{n}$ is asymptotically linear

[TABLE]

Here

[TABLE]

where $\gamma$ denotes Euler’s constant, $\psi(z)$ is the digamma function and $\chi_{k}$ is defined in (2).

Remark 1*.*

If $\frac{\log p}{\log q}\not\in\mathbb{Q}$, then only $k=0$ is needed and the second term (the sum over $j$) on the right-hand side of (11) has to be dropped. Also the first term here $\frac{\Gamma(\chi_{k})}{h}\left(1-\frac{\chi_{k}+2}{2^{\chi_{k}+1}}\right)$ is taken to be its limit $\frac{1}{h}(\log 2+\frac{1}{2})$ as $\chi_{k}\to 0$ when $k=0$.

The asymptotic estimate for the correlation coefficient in Theorem A now follows from this and the results for the variances of $S_{n}$ and $K_{n}$ (see Table 1), where expressions for $g^{(1)}_{k}$ and $g^{(3)}_{k}$ can be found, e.g., in [10]. For convenience, we give below the expressions in the unbiased case. Note that both $\mathscr{F}[g^{(1)}](n)$ and $\mathscr{F}[g^{(3)}](n)$ are strictly positive; see Schachinger [26] for details.

In the symmetric case, an alternative expression to (11) (avoiding the convolution of two Fourier series) is

[TABLE]

see the discussion of the size of tries in [10], where a similar alternative expression was given for $g^{(1)}_{k}$, which reads

[TABLE]

Moreover, also in [10], the following expression for $g^{(3)}_{k}$ can be found

[TABLE]

Note that $\chi_{k}=\frac{2k\pi i}{\log 2}$ and $2^{\chi_{k}}=1$, and the reason of retaining $2^{\chi_{k}+2}$ in the denominator is to give a uniform expression for all $k$ (notably $k=0$). These provide an explicit expression for the periodic function $F(n)$ in Theorem A. Also, since all the periodic functions have very small amplitude, the average value of the periodic function $F(z)$ can be well-approximated by

[TABLE]

3 Limit Law

In this section, we prove Theorem B, part (i); the proof of part (ii) is similar and only sketched. The key tool of the proof is the multivariate version of the contraction method; see Neininger and Rüschendorf [23]. More precisely, we will use Theorem 3.1 in [23].

We first recall the expression for the square-root of a positive-definite $2\times 2$ matrix

[TABLE]

It is well-known that such a matrix has exactly one positive-definite square root which is given by

[TABLE]

with the inverse

[TABLE]

Now we give the proof of Theorem B, part (i).

Proof of Theorem B, Part (i).

Note first that

[TABLE]

where the notation is as in Section 1. The contraction method was specially developed for obtaining limiting distribution results for such recurrences; see [23].

We need some notation. First, define

[TABLE]

This matrix is clearly positive-definite for all $n$ sufficiently large. Next define

[TABLE]

and

[TABLE]

where $\mu(n)={\mathbb{E}}(S_{n})$ and $\nu(n)={\mathbb{E}}(K_{n})$.

Now to apply the contraction method in [23], it suffices to show that the following conditions hold

[TABLE]

for $i=1,2$ and $j\in\mathbb{N}$, where $\stackrel{{\scriptstyle L_{3}}}{{\longrightarrow}}$ denotes convergence in the $L_{3}$-norm, $\|\cdot\|_{\text{op}}$ is the operator norm, $\chi_{S}$ denotes the characteristic function of set $S$, $B_{n}^{(1)}=B_{n},B_{n}^{(2)}=n-B_{n}$ and

[TABLE]

Then the contraction method in [23] guarantees that $(S_{n},K_{n})$ (centralized and normalized) converges in distribution to the unique fixed-point with mean [math], covariance matrix the unity matrix and finite $L_{3}$-norm of

[TABLE]

where $(X_{1}^{*},X_{2}^{*})$ is an independent copy of $(X_{1},X_{2})$. Obviously, the bivariate normal distribution is the solution. All this is summarized as follows.

Proposition B**.**

The following convergence in distribution holds:

[TABLE]

Proof.

We only check (15) because the second condition of (16) follows along similar lines and the first condition of (16) follows from (15) in view of

[TABLE]

We start with proving (15) for $b_{n}^{(i)}$ for which we use the notations

[TABLE]

and

[TABLE]

Also define

[TABLE]

Then, by (12), we see that

[TABLE]

and a similar expression for $b_{n}^{(2)}$ holds. From the normality of both $S_{n}$ and $K_{n}$ (proved for $S_{n}$ via the contraction method in [11] and a similar method of proof also applies to $K_{n}$), we have

[TABLE]

and

[TABLE]

Moreover, we have

[TABLE]

and

[TABLE]

where $g^{(1)},g^{(2)}$ and $\lambda$ are as above. Thus, both sequences are bounded and, consequently, we obtain the claimed result with $L_{3}$-convergence above. Similarly, one proves (15) for $b_{n}^{(2)}$.

Next, we consider $M_{n}^{(i)}$. Here, we only show the claim for the $(1,1)$ entry of $M_{n}^{(1)}$ (denoted by $M_{n}^{(1)}(1,1)$) all other cases being treated similarly. First, observe that by definition and matrix square-root, we have

[TABLE]

Now, from the strong law of large numbers for the binomial distribution

[TABLE]

and from Taylor series expansion (note that all periodic functions are infinitely differentiable), we have

[TABLE]

and

[TABLE]

Thus, $M_{n}^{(1)}(1,1)\stackrel{{\scriptstyle\text{a.s.}}}{{\longrightarrow}}\sqrt{p}$ from which the claim follows by the dominated convergence theorem.

Next, set

[TABLE]

Then, we have the following simple lemma.

Lemma 1**.**

We have, as $n\rightarrow\infty$,

[TABLE]

Proof. This follows by a straightforward computation using the expressions for the matrix square-root (12) and its inverse (13). For example, the entry $(1,2)$ of $\widehat{\Sigma}_{n}^{-\frac{1}{2}}\widetilde{\Sigma}_{n}^{\frac{1}{2}}$ (where we use the notations from the proof of the previous proposition) satisfies

[TABLE]

which tends to [math] as claimed. The other entries are treated similarly,

Theorem B, part (i) now follows from this lemma and Proposition B.

Next, we sketch the (similar) proof of Theorem B, part (ii).

Proof of Theorem B, Part (ii).

The proof runs along similar lines as in Part (i). The only difference is that now it is not entirely obvious that $\widehat{\Sigma}_{n}$ is positive definite. Note, however, that from the discussion in the introduction, this matrix is positive-definite if and only if $\Sigma_{n}$ (defined in Theorem B) is positive definite. This is ensured by the following lemma.

Lemma 2**.**

$\Sigma_{n}$* is positive-definite for all $n$ large enough.*

Proof. It suffices to show that $\det(\Sigma_{n})>0$ for all $n$ large enough. Indeed, we have

[TABLE]

from which the result follows.

Note that this in addition shows the stronger result $\det(\Sigma_{n})\geqslant dn^{2}$ for all $n$ large enough where $d>0$. (A proof avoiding numerical computations can be performed using the same approach as in Proposition 3 of [12].)

The rest of the proof is similar as in the asymmetric case and is omitted.

4 Extensions

In this section, we show that the dependence phenomena we discovered here on random binary tries (Theorem A and Theorem B) also find their appearance in other trees and structures whose subtree-sizes and sub-structure-sizes are dictated by a binomial or a multinomial distribution.

For simplicity, we consider in this section only three varieties of random digital trees: random $m$-ary tries, random PATRICIA tries and random bucket digital search trees; see [10] for more potential examples with the same splitting principles.

$m$-ary Tries.

It is straightforward to extend our tries constructed from binary input strings to inputs from an $m$-ary alphabets, $m\geqslant 2$. In this case, the resulting trie becomes an $m$-ary tree (since each node now has $m$ subtrees one belonging to each letter). As a random model, we assume that bits are generated independently at random with the $i$-th letter occurring with probability $p_{i}$, where $p_{1}+\cdots+p_{m}=1$ and $0<p_{i}<1$ for $1\leqslant i\leqslant m$.

The size and the key path length (which we again denote by $S_{n}$ and $K_{n}$) in such random $m$-ary tries satisfy the recurrences

[TABLE]

with the initial conditions $S_{n}=K_{n}=0$ for $n\leqslant 1$, where $(S_{n}^{(i)})$ and $(K_{n}^{(i)})$ are independent copies of $(S_{n})$ and $(K_{n})$, respectively, for $1\leqslant i\leqslant m$, and

[TABLE]

for all $j_{1},\ldots,j_{m}\geqslant 0$ with $j_{1}+\cdots+j_{m}=n$.

The pair $(S_{n},K_{n})$ satisfies the same type of properties as those described in Theorem A and Theorem B for binary tries, where the symmetric case here corresponds to $p_{1}=\cdots=p_{m}=1/m$ and all other cases are asymmetric. Only the expressions for $g^{(1)}_{k},g^{(2)}_{k},g^{(3)}_{k}$ and $\lambda$ are different but they can be computed via the same analytic tools as those used in [10]. For the sake of simplicity, we only give the expressions in the symmetric case ($\chi_{k}=2k\pi i/\log m$) as follows:

[TABLE]

Note that the variance of the size was considered in [12], but no explicit expression was given for the Fourier coefficients of the periodic function.

With the help of these expressions, we obtain the following numerical approximations to the average value of the periodic function of the correlation coefficient between $S_{n}$ and $K_{n}$ in the symmetric case. We see that they differ little.

PATRICIA Tries.

A simple idea to increase the efficiency of tries is to remove all internal nodes with one-way branching. The resulting tree is called a PATRICIA trie; here PATRICIA is an acronym of “Practical Algorithm To Retrieve Information Coded In Alphanumeric”.

We use the same random model as we used above for $m$-ary tries and consider the size and key-path length of PATRICIA tries (which we again denote by $S_{n}$ and $K_{n}$). Then they satisfy the recurrences

[TABLE]

with the initial conditions $S_{n}=K_{n}=0$ for $n\geqslant 1$, where $I_{n}^{(i)}$ is defined as in (17) above, $(S_{n}^{(i)})$ and $(K_{n}^{(i)})$ are independent copies of $(S_{n})$ and $(K_{n})$, respectively, and

[TABLE]

Note that for $m=2$, the size is deterministic. We thus assume $m\geqslant 3$ to avoid trivialities. Then the dependence of $(S_{n},K_{n})$ satisfies mutatis mutandis Theorem A and Theorem B. In particular, the required changes for $g^{(1)}_{k},g^{(2)}_{k},g^{(3)}_{k}$ in the symmetric case are given as follows ($\chi_{k}=2k\pi i/\log m$):

[TABLE]

These expressions are also valid for $m=2$, where $g^{(1)}_{k}$ and $g^{(2)}_{k}$ can be shown to be identically zero. Note that the result for the variance of the key-path length was already derived in [10] (for $m=2$) and that for the size was established in [12] but without a precise expression for the Fourier coefficients.

Again, we can use the above expressions to obtain the average value of the periodic function of the correlation coefficient between $S_{n}$ and $K_{n}$ in the symmetric case. Note that unlike tries, these values increase with $m$.

Bucket Digital Search Trees.

Digital search trees (DST) represent yet another class of digital tree structures; see [18, 19] for more information. In contrast to tries and PATRICIA tries, they only have one type of nodes where data are stored. More precisely, given a set of data consisting of $n$ infinite [math]-$1$ strings, a DST is constructed as follows: if $n=1$, then the DST consists of only one node holding the sole string; otherwise, the first string is stored in the root and all others are directed to the subtrees according to their first bit being [math] or $1$; then, the subtrees are built recursively but by using consecutive bits to split the data.

Clearly, the size of such a DST is deterministic and equals the input cardinality. We consider instead a bucket version with an additional capacity $b\geqslant 2$, allowing each node holding up to $b$ strings and nodes having subtrees only when they are filled up.

We adopt the same Bernoulli random model as for random tries and consider the size and key-path length in random bucket digital search trees (again denoted by $S_{n}$ and $K_{n}$), which then satisfy

[TABLE]

with the initial conditions $S_{0}=K_{0}=K_{1}=\cdots=K_{b-1}=0$ and $S_{1}=\cdots=S_{b-1}=1$, where $(S_{n}^{*})$ and $(K_{n}^{*})$ are independent copies of $(S_{n})$ and $(K_{n})$, respectively.

The same dependence phenomena as those described in Theorem A and Theorem B also hold for the pair $(S_{n},K_{n})$. The computation of the sequences $g^{(1)}_{k},g^{(2)}_{k},g^{(3)}_{k}$ is nevertheless more intricate. In the asymmetric case, one can again use analytic de-Poissonization and Mellin transform techniques, however, the resulting expressions are less explicit. On the other hand, in the symmetric case, explicit expressions for $g^{(1)}_{k},g^{(2)}_{k},g^{(3)}_{k}$ are available via the Poisson-Laplace-Mellin method from [13]. As the expressions are long, we omit them here. Note that the results for the variances of $S_{n}$ and $K_{n}$ have already been obtained in [13].

Other Shape Parameters.

Theorem A and Theorem B also extend to pairs of random variables where the size is replaced by the number of various patterns (such as the number of internal-external nodes discussed, e.g., by Flajolet and Sedgewick in [8]) and the key-path length is replaced by other notions of the path length (such as the total path length of internal-external nodes).

Acknowledgments

The first author acknowledges partial supported by MOST under the grants MOST-104-2923-M-009-006-MY3 and MOST-105-2115-M-009-010-MY2. We also thank the helpful comments by the referees for the extended abstract [9] of this paper.

Appendix A A Sketch of Proof of (11)

We sketch here some details of how the expression (11) for $g^{(2)}_{k}$ in Proposition A is obtained. The method we use is based on that introduced in [10]; see p. 25 et seq.

First, by moving the line of integration in (10) to the right and using the residue theorem, we have

[TABLE]

where $G_{2}(s)={\mathscr{M}}[\tilde{h}_{1}(z)+\tilde{h}_{2}(z);s]$. Note that in the above expression and in what follows, if $\log p/\log q$ is irrational, then only the term with $k=0$ is retained. Thus, our problem boils down to the computation of $G_{2}(s)$.

We first consider the Mellin transform of $\tilde{h}_{2}(z)$ which is easier to handle. By the expression (9) for $\tilde{h}_{2}(z)$ from Section 2, the Mellin transform is given by

[TABLE]

where

[TABLE]

Observe that by applying the Mellin transform and its inverse to (5), we obtain

[TABLE]

and

[TABLE]

Substituting these into the integral representation of $Y(s)$ and interchanging the integrals, we see that

[TABLE]

where the last line follows from moving the vertical line of integration to minus infinity and summing over all the residues of the poles encountered.

For ${\mathscr{M}}[\tilde{h}_{1}(z);-1+\chi_{k}]$, we use the expression for $\tilde{h}_{1}(z)$ in Section 2, (18) and (19), and Mellin convolution, giving

[TABLE]

where the integration path is the imaginary axis with a small indentation to the right at the zeros of $1-p^{1-\omega}-q^{1-\omega}$. Now by the decomposition

[TABLE]

the above integral is rewritten as

[TABLE]

We break now this integral into two parts according to the two terms in the bracket. For the first part, we use the substitution $\omega\leftrightarrow\chi_{k}-\omega$ and standard residue calculus, and obtain

[TABLE]

where the second line follows by moving the line of integration over the imaginary axis and $\psi(s)$ denotes the derivative of $\log\Gamma(s)$. Next, note that

[TABLE]

The first integral on the right-hand side is a Mellin convolution integral and can be evaluated explicitly as

[TABLE]

For the second integral, we move the line of integration to infinity and use the residue theorem, yielding

[TABLE]

In a similar way, the second part of (20) has the series representation

[TABLE]

Since

[TABLE]

we then deduce (11) by collecting all expressions.