On Universal Codes for Integers: Wallace Tree, Elias Omega and Variations

TL;DR

This paper introduces and analyzes the Wallace Tree Code (WTC), a universal integer coding method based on binary trees, comparing it to Fibonacci and Elias omega codes, with proposed improvements to Elias omega.

Contribution

It presents the encoding and decoding routines for WTC, analyzes its properties, and proposes enhancements to Elias omega code, advancing universal integer coding techniques.

Findings

WTC offers competitive compression performance

WTC's properties are comparable to Fibonacci and Elias omega codes

Proposed improvements enhance Elias omega code efficiency

Abstract

A universal code for the (positive) integers can be used to store or compress a sequence of integers. Every universal code implies a probability distribution on integers. This implied distribution may be a reasonable choice when the true distribution of a source of integers is unknown. Wallace Tree Code (WTC) is a universal code for integers based on binary trees. We give the encoding and decoding routines for WTC and analyse the properties of the code in comparison to two well-known codes, the Fibonacci and Elias omega codes. Some improvements on the Elias omega code are also described and examined.

Tables

Table 1. TABLE I: Examples of code-words

N	Fib	Elias $\omega$	WTC1
1	11	1	0
2	011	010	100
3	0011	011	10100
4	1011	000100	11000
5	00011	000101	1010100
6	10011	000110	1011000
7	01011	000111	1100100
8	000011	0011000	1101000
9	100011	0011001	1110000
10	010011	0011010	101010100
11	001011	0011011	101011000
12	101011	0011100	101100100
13	0000011	0011101	101101000
14	1000011	0011110	101110000
15	0100011	0011111	110010100
16	0010011	00000010000	110011000
17	1010011	00000010001	110100100
18	0001011	00000010010	110101000
19	1001011	00000010011	110110000
20	0101011	00000010100	111000100
21	00000011	00000010101	111001000
22	10000011	00000010110	111010000
23	01000011	00000010111	111100000
24	00100011	00000011000	10101010100
100	00101000011	0000101100100	1011101001000
Below code-word lengths are shown, instead of code-words
${10}^2$	$\left|CW\right|=11$	$\left|CW\right|=13$	$\left|CW\right|=13$
${10}^3$	$\left|CW\right|=16$	$\left|CW\right|=17$	$\left|CW\right|=17$
${10}^4$	$\left|CW\right|=20$	$\left|CW\right|=21$	$\left|CW\right|=21$
${10}^5$	$\left|CW\right|=25$	$\left|CW\right|=28$	$\left|CW\right|=25$
${10}^6$	$\left|CW\right|=30$	$\left|CW\right|=31$	$\left|CW\right|=27$
${10}^7$	$\left|CW\right|=35$	$\left|CW\right|=35$	$\left|CW\right|=31$
${10}^8$	$\left|CW\right|=39$	$\left|CW\right|=38$	$\left|CW\right|=35$
${10}^9$	$\left|CW\right|=44$	$\left|CW\right|=41$	$\left|CW\right|=39$
googol	$\left|CW\right|=480$	$\left|CW\right|=349$	$\left|CW\right|=345$

Table 2. TABLE II: Cumulative probabilities up to code-word length | w ​ ( N ) | 𝑤 𝑁 |w(N)| .

$\left|w\left(N\right)\right|$ (bits)	Fibonacci	Elias $\omega$	WTC1
1	0	0.5	0.5
2	0.25	0.5	0.5
3	0.375	0.75	0.625
4	0.5	0.75	0.625
10	0.859	0.875	0.754
100	0.999…	0.947	0.920
1000	0.999…	0.957	0.975
10000	0.999…	0.963	0.992
100000	0.999…	0.9688	0.997
1000000	0.999…	0.9692	0.9992

Table 3. TABLE III: Code-word lengths (in bits) for varying N 𝑁 N (chosen to highlight early points of change) across Fibonacci, Elias and WTC1. Shortest code-word lengths for a given N 𝑁 N are asterisked

N	Fibonacci	Elias $\omega$	WTC1
1	2	1*	1*
2	3*	3*	3*
3	4	3*	5
4	4*	6	5
13	7*	7*	9
16	7*	11	9
610	15*	17	15*
627	15*	17	17
1597	17*	18	17*
2057	17*	19	19
4181	19*	20	19*
6765	20	20	19*
6919	20*	20*	21
8192	20*	21	21
10946	21*	21*	21*
16384	21*	22	21*
17711	22	22	21*
23715	22*	22*	23
28657	23	22*	23
32768	23*	23*	23*
46368	24	23*	23*
65536	24	28	23*
82501	25*	28	25*

Equations

$$H=\sum _{\forall N>0}\mathrm{Pr}(N)\log \left(\frac{1}{\mathrm{Pr}(N)}\right)$$

$$E_w=\sum _{\forall N>0}\mathrm{Pr}(N)|w(N)|<K\cdot H,$$

$$\frac{E_w}{\max (1,H)}\le R(H)\le K,$$

$$N=36⟹\text{trim}(\text{omega}(36))$$

$$\text{\#sections}=3⟹\text{trim}(\text{omega}(3))$$

$$\text{\#sections}=1⟹\text{trim}(\text{omega}(1))$$

$$\text{\# tetrations}=3.$$

$$=$$

$$⟹{\text{omega}}^{\ast }(36)=011100100100$$

$$C_f=\frac{1}{f+1}\left(\genfrac{}{}{0ex}{}{2f}{f}\right)=\frac{(2f)!}{(f+1)!f!}.$$

$$\begin{array}{rl}{\mathrm{paths}}_{r,c}& =0,\text{ if }c>r,\\ {\mathrm{paths}}_{0,0}& =1,\\ {\mathrm{paths}}_{r,0}& =1,r\ge 0,\\ {\mathrm{paths}}_{r,c}& ={\mathrm{paths}}_{r-1,c}+{\mathrm{paths}}_{r,c-1},\text{ otherwise}.\end{array}$$

$$1)$$

$$2)$$

$$L(N)=2f(N)+1$$

$$f(N)=\underset{f}{\inf }\left\{\mathbb{Z}:N>c{C}_f\right\}$$

$$\overline{f}(N)\equiv \overline{f}=\frac{\left(\log N+\frac{3}{2}\log \left(\frac{\log N}{\log 4}\right)+\frac{1}{2}\log \left(\frac{9\pi }{16}\right)\right)}{\left(1-3/2/\log N\right)\log 4}$$

$$\underline{f}(N)\equiv \underline{f}=\frac{\log N}{\log 4-(3/2/\overline{f})\log \overline{f}}.$$

$$2\underline{f}(N)+1<L(N)<2\overline{f}(N)+1.$$

$$\text{noindentTheorem 1.}L(N)={\log }_{2}N+\frac{3}{2}{\log }_2{\log }_{2}N+{\epsilon }_N,$$

$$\underset{N\to \infty }{\mathrm{limsup}}\{{\epsilon }_N\}$$

$$\underset{N\to \infty }{\mathrm{liminf}}\{{\epsilon }_N\}$$

$$L(N;c)=\left\{\begin{array}[]{ll}1&\;\mbox{for }N=0\\ 3&\;\mbox{for }N=1\\ \log_{2}N+\frac{3}{2}\log_{2}\log_{2}N+c&\;\mbox{for }N\geq 2\end{array}\right.$$

$$c{C}_f>\frac{4^{f+1}}{3(f+1)\sqrt{\pi f}}.$$

$$(f+1)\log 4-\frac{1}{2}\log 9\pi -\log (f+1)-\frac{1}{2}\log f=\log N.$$

$$-\log (f+1)-\frac{1}{2}\log f$$

$$\log c{\underline{C}}_f=f\log 4-\frac{3f}{2f_0}-\frac{3}{2}\log f_0-\frac{1}{2}\log \left(\frac{9\pi }{16}\right).$$

$$\overline{f}(N;f_0)=\frac{\log N+\frac{3}{2}\log f_0+\frac{1}{2}\log \left(\frac{9\pi }{16}\right)}{\log 4-3/2/f_0}.$$

$$c{C}_f<\frac{4^{f+1}}{3\sqrt{\pi f^3}}.$$

$$(f+1)\log 4-\frac{1}{2}\log 9\pi -\frac{3}{2}\log f=\log N.$$

$$-\frac{3}{2}\log f<-\left(\frac{3\log f_1}{2f_1}\right)\min \{f,f_1\}$$

$$\log c{\overline{C}}_f=f\log 4-\left(\frac{3\log f_1}{2f_1}\right)\min \{f,f_1\}.$$

$$\underline{f}(N)\equiv \underline{f}=\frac{\log N}{\log 4-(3/2/\overline{f})\log \overline{f}}$$

$$L_A(N)={\log }_{2}N+\frac{3}{2}{\log }_2{\log }_{2}N,$$

$$\overline{f}=\frac{L_A(N)+\frac{1}{2}{\log }_2\left(\frac{9\pi }{128}\right)}{2\left(1-3/2/\log N\right)}$$

Full text

I Introduction

Universal codes for the positive integers, $N\in\mathbb{Z}^{+}$, are of interest for at least three reasons. The first is in everyday data compression where such a code can be used to store or to transmit a sequence of integers when their true probability distribution is not known. The second is in inductive inference, where a countable set of hypotheses in a statistics and machine learning task is mapped to the set of positive integers: If the true distribution of the set of hypotheses in an inference problem is unknown but they can be plausibly ordered in non-increasing probability, then the $i^{\text{th}}$ hypothesis can be assigned the probability $2^{-|w(i)|}$, where $w(i)$ is the code-word of integer $i$, whose length is represented as $|w(i)|$. Finally, it must also be admitted that there is simply a fascination in trying to devise an efficient code for truly enormous integers.

Elias [1] defined a code having the universal property as one where the code-word length is monotonically increasing and “assigning messages in order of decreasing probability to codewords in order of increasing length gives an average code-word length, for any message set with positive entropy, less than a constant times the optimal average codeword length for that source.” If the source has distribution $\Pr(\cdot)$ and entropy

[TABLE]

then, for any universal code $w(\cdot)$ for positive integers,

[TABLE]

where $K$ is a constant independent of $\Pr(\cdot)$. The latter sum is at least finite, although the distribution implied by a universal code must itself have infinite entropy. Naturally we hope that $K$ is not large. Elias also defined an asymptotically optimal code as one where the ratio

[TABLE]

where $R$ is a function of $H$ with $\lim_{H\rightarrow\infty}R(H)=1.$

Wallace proposed a universal code for integers [2] (WTC) inspired by binary trees. He suggested that its implied probability distribution is a reasonable choice to use when the true distribution of a source of integers is unknown. In the following sections, the WTC, Fibonacci and Elias omega codes are summarised. Encoding/Decoding routines and asymptotic analysis are given for WTC. The properties of the three codes, particularly the lengths of code-words, are compared. Ways of improving the Elias omega code for large integers are also discussed.

Note, if we have non-negative integers, a code for $N\geq 1$ can be shifted and used for $N\geq 0$ by adding one before encoding and subtracting one after decoding. If we have all of the integers to deal with, they can be ordered as $[0,1,-1,2,-2,3,\ldots]$, say, and $N$ can be encoded according to its position in the list. Encoding and decoding routines for the codes described can be found, and interactively experimented with, at www.allisons.org/ll/MML/Discrete/Universal/.

II Three universal codes

The Fibonacci and Elias omega codes for positive integers are described below for later comparison to the Wallace tree code (WTC) for positive integers. The main focus is in the Elias omega code and WTC; the Fibonacci code is included as a fixed point of comparison to the other codes.

II-A Fibonacci code for integers

The Fibonacci code [3] is based on the Fibonacci numbers $F_{0}=1,F_{1}=1,\text{~{}and~{}}F_{j}=F_{j-1}+F_{j-2},\text{~{}for~{}}j\geq 2$, which are $[1,1,2,3,5,8,\ldots]$. The code ignores $F_{0}$ and uses $F_{1}$ and upwards. To encode an integer $N\geq 1$ do as follows:

• •

Find the largest $F_{j_{1}}$ that is less than or equal to $N$ and remember $j_{1}$.

• •

Repeat for $N^{\prime}=N-F_{j_{1}}$, finding $j_{2},j_{3}$, and so on until zero remains.

We have $N=F_{j_{1}}+F_{j_{2}}+\ldots$. The code-word for $N$ consists of $j_{1}+1$ bits. If $F_{k}$ appears in the sum for $N$ set the $k^{\text{th}}$ bit of the code-word to ‘1’ otherwise set it to ‘0’. Set the last bit to ‘1’. It is easy to see that the encoding is well defined and that each $j_{k}<j_{k-1}-1$. Note that “11” cannot appear in the code-word except at the very end and that a code-word of length $|w(N)|$ contains at least two and up to $2+\lfloor(\frac{|w(N)|-2)}{2}\rfloor$ ‘1’s.

II-B Elias omega ($\omega$) code for integers

We first introduce the following variation on the Elias omega code that gives identical code-word lengths to the original definition [1] and differs in only minor details. The code-word for an integer $N\geq 1$ consists of one or more sections: zero or more length sections followed by one value section. The value section is simply the usual binary representation of $N$ in $\lfloor\log_{2}(N)\rfloor+1$ bits; note that the value section starts with a ‘1’.

The code-word for $N=1$ is simply “1”; it is the only code-word that starts with a ‘1’. The code-word for $N\geq 2$ has at least one length section before the final value section. In general the value section by itself is not sufficient because a decoder does not know how long it is when $N\geq 2$. The solution is to first encode the length of the value section minus one (the length must be $\geq 2$ when $N\geq 2)$, recursively, until the length$-1$ of a length$-1$ of $\ldots$ of a length$-1$ gives one.

The leading bit of each section would, on the face of it, be a ‘1’ so that position can instead be used as a flag to indicate either a length section (‘0’) or the final value section (‘1’). The decoder notes the flag. In the case of a ‘0’ it then switches it to a ‘1’ before computing the length of the next section. If present ($N\geq 2$) the first length section is just “0” which stands for one. If $N\geq 4$ the second length section is either “00” which stands for two or “01” which stands for three, and so on.

Note that the omega code is very similar to, and can be thought of as an optimized and shifted version of, the Levenstein code [4] which is defined for $N\geq 0$. Also note that Rissanen [5] defined the $\log^{*}$ code as an approximation to the omega code; $\log^{*}(N)=c+\log_{2}(N)+\log_{2}(\log_{2}(N))+\log_{2}(\log_{2}(\log_{2}(N)))\ldots$, all positive terms, where $c$ is a normalising constant, and $\Pr(N)=2^{-\log^{*}(N)}$.

II-B1 The omega variations

The Elias omega code in effect uses a unary code (“0…” $\Rightarrow$ length section, “1…” $\Rightarrow$ final value section) to indicate the number ($\geq 1$) of sections in a code-word. Elias chose the name omega for the code because he considered it to be “penultimate”, that is “not quite ultimate” (p.200)[1]. (That being so, the name psi, say, would have left some room to name codes that are closer to the ultimate.) He noted that the unary code could be replaced with his gamma, delta or omega codes.

In fact the leading bits of the sections can be moved to the front of the code-word and the unary code can be replaced by any other code (universal or not) for positive integers – say Fibonacci or WTC.

Define omegap(s) to be the Elias omega code modified and parameterised to use an integer code ‘s’ for the number of sections. The code-word for $1$ is “1”. For $N>1,$ the code-word is the omega code-word for $N$ with the leading bit of every section trimmed away, and the result preceded by the ‘s’ code-word for the number of sections. omegap(unary) is equivalent to the usual omega code. The code omegap(Fibonacci) would make code-words of one, two and three sections longer and those of six or more sections shorter than under omega. Let omega2 = omegap(omega) which uses the omega code for the number of sections. This code would make code-words of two, four and five sections longer and those of seven or more sections shorter than the usual Elias omega code.

Even omega2 is not ultimate. It is possible to define a code, omegar(t), that uses itself, recursively, to state the number of sections (minus one) in an omega code-word. Note that omegar($\cdot$) needs some other integer code to encode the number of tetrations, i.e. the number of times that omegar is applied. Let omega* = omegar(omega). For example, $N=36$ is encoded using omega* as follows (spaces added in code-words below for readability):

[TABLE]

where $\text{trim}(.)$ removes the leading bit of each length and value section of recursively applied omega code-words.

Comparing against the omega code, omega* would make code-words of integers with nine or more sections shorter than under omega and the smallest corresponding integer is $2\uparrow\uparrow 8$ in Knuth’s up-arrow notation. An encoder for omegar is given in  Fig. 1.

Even omega* is not ultimate because one can reconsider the encoding of the number of tetrations but the integers for which there would be any further improvement would be “very large” indeed.

II-C Wallace tree code (WTC) for integers

The Wallace tree code for integers [2][6] are based on full binary trees, and hence depend on the Catalan numbers $C_{f},\forall f\geq 0$, with the first few numbers being $C_{0}=1,C_{1}=1,C_{2}=2,C_{3}=5,C_{4}=14,$ and so on.

Any full binary tree consists of $f\geq 0$ fork (i.e. internal) nodes and $f+1$ leaf nodes. Each fork node has exactly two sub-trees and each leaf node has zero sub-trees.

The number of full binary trees containing $f$ fork nodes is the $f^{\text{th}}$ Catalan number, defined as:

[TABLE]

This yields the recurrence $C_{f+1}=\frac{2(2f+1)C_{f}}{f+2}$. Furthermore, $\lim_{f\rightarrow\infty}\frac{C_{f+1}}{C_{f}}=4,$ and $C_{f+1}=\sum_{j=0}^{f}(C_{j}\cdot C_{f-j})$ [7]. Finally, it is useful to define the cumulative Catalan numbers $cC_{f}=\sum_{j=0}^{f}C_{f},\forall f\geq 0$, which are $cC_{0}=1,cC_{1}=2,cC_{2}=4,cC_{3}=9,cC_{4}=23,$ and so on.

As a preliminary matter, note that a full binary tree can be encoded [8] during a prefix traversal of the tree, outputting a ‘1’ for each fork and a ‘0’ for each leaf (e.g. see  Fig. 2). The end of the tree’s code-word is indicated upon reaching one more ‘0’ than ‘1’s, and this event does not happen earlier within the code-word so this is a prefix code. It is easy to recover the tree from the code-word.

Reading a ‘1’ as left and a ‘0’ as down, a tree’s code-word can also be interpreted as encoding a Dyck path [9] in a square lattice from some, initially unknown, point on the diagonal $(f,f)$ to $(0,-1)$, that is, a zig-zag path that does not go below the diagonal until it terminates with the final ‘0’. Fig. 3 shows three of the five paths from $(3,3)$ to $(0,-1)$ with their code-words including the lexicographically first ‘1010100’ and last ‘1110000’. The number of paths (Fig. 4) from row $r$, column $c$, to $(0,0)$ is given by

[TABLE]

It can be shown that $\mathrm{paths}_{f,f}=C_{f}$.

The code for integers is most easily explained for integers $N\geq 0$; call this version of the code WTC0. The full binary trees are sorted on their code-word lengths and, for a given length, lexicographically. For a given length, the first code-word is of the form $(10)^{f}0$ and the last $1^{f}0^{f+1}$. Integer $N\geq 0$ is given the code-word of the $N^{th}$ full binary tree (in the lexicographic order of full binary trees, counting from zero).

Encoding and decoding routines are given in  Fig. 5. The code-word for $N=0$ is “0”. For $N>0$, the $C_{f}$ integers in the range $cC_{f-1}<N\leq cC_{f}$, all have code-words of length $2f+1$. $f$ can be found by searching for the largest cumulative Catalan number, $cC_{f-1}$, that is less than $N$. The code-word for $N$ is the $K^{th}$ of the lexicographically ordered code-words of length $2f+1$ where $K=(N-cC_{f-1})$. The code-word can be found using $\mathrm{paths}_{r,c}$. Starting with an empty code-word at position $(r,c)$ where $r=f,c=f$. If $K>\mathrm{paths}_{r-1,c}$ then append a ‘1’ (move left, $c=c-1$) to the code-word and we need a code-word at least that much further up the rankings ($K=K-\mathrm{paths}_{r-1,c}$). Otherwise, append a ‘0’ (move down, $r=r-1$). Repeat until $r=c=0$.

The decoding routine (also in Fig. 5) follows similar logic to the encoding routine. A valid code-word, $str$, contains $f=\lfloor|str|/2\rfloor$ ‘1’s and $f+1$ ‘0’s and no proper prefix contains more ‘0’s than ‘1’s. As $str$ is processed from left to right, every ‘1’ (move left) means that $str$ is known to be at least $\mathrm{paths}_{r,c}$ further up in rank amongst the code-words of this length. Repeat until the end of the code-word.

As noted before, it is easy to “shift” WTC0 code (which encode integers $N\geq 0$) to instead encode integers $N\geq 1$. Call the shifted code WTC1 if we need to distinguish between it and WTC0.

II-D Examples

Table I gives examples of integers coded under the three codes. Note that the lengths of Fibonacci code-words increase in steps of one from time to time as $N$ grows. The lengths of WTC1 code-words increase in steps of two. Lengths in the Elias omega code increase in steps of various sizes, for example increasing by four on going from $N=15$ to $N=16$ when the value section grows by one and a whole new length section is added; there is no upper limit to the step size.

III Implied probability distributions

An efficient code for the positive integers $N\geq 1$ implies a probability distribution on them in which $\Pr(N)=2^{-|w(N)|}$. The Fibonacci, Elias omega and Wallace tree (WTC1) codes all imply proper probability distributions on the positive integers: Consider an infinite string of bits generated at random, independent and identically distributed (i.i.d.), with $\Pr(\text{`0'})=\Pr(\text{`1'})=0.5$. For each code, the infinite string has some prefix, of length $L$ and probability $\frac{1}{2^{L}}$, which is a valid code-word in that code. In principle, by removing the prefix and repeating forever, all possible code-words will be sampled in proportion to their probabilities under the corresponding distribution.

III-A Fibonacci:

• •

There must be a first occurrence of “11” in the infinite string. It marks the end of a prefix of length $L\geq 2$ which is a valid code-word in the Fibonacci code. The probability of the prefix is $2^{-L}$. There are $i=L-2$ positions before the final “11”. Each of these position can hold ‘0’ or ‘1’ but no two adjacent positions, other than the last, can both hold ‘1’.

• •

There are $F_{i}$ code-words of length $L=i+2$, $i\geq 0$ (a code-word of length $L$ can be formed by prepending one of length $L-1$ with a ‘0’, or prepending one of length $L-2$ with a “10”).

• •

The total probability of those integers having code-words of length $(i+2)$ is $\frac{F_{i}}{2^{i+2}},i\geq 0$.

• •

The total probability of all positive integers, $\sum_{i\geq 0}\frac{F_{i}}{2^{i+2}}$, must be one.

III-B Elias omega:

• •

An Elias code-word is made up of zero or more length sections followed by one value section. The infinite string of bits starts either ‘1’ or ‘0’. If it starts ‘1’, that itself is a prefix which is an Elias code-word for the value one, of probability $0.5$. If it starts ‘0’ that is decoded as $1$ (the lead bit having been changed) which indicates that a section of length $2=1+1$ follows. If the next section starts ‘1’ it is a value. If it starts ‘0’ it is a length, “00” giving $3=2+1$ or “01” giving $4=3+1$. And so on.

• •

Eventually a section starting ‘1’, of length $s$, will appear marking the end of a code-word of length $L$.

• •

There are $2^{s-1}$ code-words of length $L$.

III-C WTC:

• •

Because a one-dimensional random walk (‘0’   left, ‘1’   right, say) returns to the origin with probability one, the infinite string has some prefix of length $L=(2f+1),i\geq 0$, that is a valid WTC1 code-word. The probability of the prefix is $\frac{1}{2^{2f+1}}$.

• •

There are $C_{f}$ code-words of length $(2f+1)$.

• •

The total probability of those integers having code-words of length $(2f+1)$ is $C_{f}/2^{2f+1},f\geq 0$.

• •

The total probability of all positive integers, $\sum_{f\geq 0}C_{f}/2^{2f+1}$, must be one.

Table II gives cumulative probabilities for integers having code-words upto a given length. Note that an Elias omega code-word of $1,000,000$ bits needs at least six sections, the maximum lengths of sections being $1,2,4,16,65536,2^{65536},\ldots$. The probability of having five or fewer sections is $1/2+1/4+1/8+1/16+1/32=1-0.03125=0.96875$, slightly less than 0.9692.

IV Comparative code-word lengths

For integers less than $100,000$ the “lead”, in the sense of having the shortest code-words, changes hands between the three codes but is often held by the Fibonacci code (table III) until $N=317,811$ where it falls out of contention. Beyond that, and up to the $506$ decimal digit integer corresponding to $cC_{847}+1$, WTC has the shortest code-words, rarely equalled by the Elias omega code (i.e., for values between $cC_{134}+1$ and $2^{255}-1$, both codes using $269$ bits). To contextualize these numbers, this is past the size of the human genome ($3.2\times 10^{9}$ base-pairs), the estimated number of baryons in the universe ($10^{80}$) and one googol ($10^{100}$). Therefore beyond some further point the Elias omega code must take the lead either permanently or at least most of the time – see section VI.

For each of the three codes, integers come in “blocks” that contain integers having code-words of the same length under that code. The sizes of the blocks differ between the codes. For WTC, the blocks are $[1,1],[2,2],[3,4],[5,9],...,[cC_{f-1}+1,cC_{f}],...$ having code-lengths $1,3,5,7,...,2f+1,...$ bits respectively. For $f\geq 848$ and up to at least $f=1000$, $|\text{WTC}(cC_{f-1}+1)|=|\text{omega}(cC_{f-1}+1)|$ and $|\text{WTC}(cC_{f})|=|\text{omega}(cC_{f})|-2$.

The smallest $f>134$ such that $|\text{WTC}(cC_{f-1}+1)|=|\text{omega}(cC_{f-1}+1)|$ is $848$. The code-length is $1697$ bits. 2. 2.

The smallest $f>134$ such that $|\text{WTC}(cC_{f-1}+1)|>|\text{omega}(cC_{f-1}+1)|$ is $3389$. The code-lengths are $6779$ and $6778$ bits, respectively. 3. 3.

The smallest $f>134$ such that $|\text{WTC}(cC_{f})|>|\text{omega}(cC_{f})|$ is $13,877,006$. The code-lengths are $27,754,013$ and $27,754,012$ bits, respectively. (There are larger $f$ where $|\text{WTC}(cC_{f})|\leq|\text{omega}(cC_{f})|$.)

V Robustness

If a code is used for storing or transmitting a sequence of integers, as opposed to calculating entropy, the effect of errors may be of interest. The addition of extra error-correcting mechanisms is not covered here. Consider a sequence of integers, $[N_{1},...,N_{k}]$, encoded in each of the codes and imagine the consequences of a bit being flipped in error.

In the Fibonacci code, switching a ‘1’ to a ‘0’ in the code-word of $N_{j}$ causes the value of $N_{j}$ to be misread unless it is one of the last two ‘1’s. In the latter case the end of $N_{j}$ is not detected correctly and it absorbs some or all of the code-word of $N_{j+1}$ depending on which bit is flipped and on whether or not $N_{j+1}$ starts with a ‘1’. If a ‘0’ is flipped to a ‘1’ and this is next to a genuine ‘1’, $N_{j+1}$ is taken to end prematurely and an extra integer is apparently inserted. A single bit error affects one or two integers and may cause an error in indexing (Fig. 6). For example:

In the Elias omega code, switching a bit of $N_{j}$’s value section causes the value to be misread unless it is the section’s leading ‘1’ that becomes ‘0’. In the latter case the section is taken to be a length and parts of one or more following code-words are mistaken as parts of $N_{j}$. If a bit in a length section of $N_{j}$ is flipped, that length is misread and too little or too much is taken for the next section unless it is the lead ‘0’ that becomes a ‘1’. In that case the length section is taken to be the value section, $N_{j}$ is taken to end prematurely, and the rest of $N_{j}$’s code-word causes further mistakes. A single bit error may affect one or many integers.

In the Wallace tree code, changing a ‘1’ to a ‘0’ in $N_{j}$ causes a premature end to the code-word (not necessarily at the change) and the remainder is taken as two extra integers. Changing a ‘0’ to a ‘1’ causes $N_{j}$ to absorb $N_{j+1}$ and $N_{j+2}$. A single bit error affects one or three integers and causes an error in indexing (fig.7). For example:

VI Asymptotic Analysis of Wallace tree code

It is of interest to determine the asymptotic behaviour of WTC for increasing integer $N$. Let $L(N)$ denote the length of the binary code-word assigned to integer $N$ by WTC1. Recall from Section II-C that $C_{f}$ denotes the $f^{\rm th}$ Catalan number, and $cC_{f}$ denotes the sum of the first $f$ Catalan numbers. Then, the length, in bits, of the code-word assigned by WTC1 to integer $N$ is

[TABLE]

where

[TABLE]

denotes the smallest integer $f$ such that $N$ exceeds $cC_{f}$.

VI-A Bounds on $L(N)$

The following lemma provides appropriate upper and lower bounds for $L(N)$.

Lemma 1. Let $L(N)$ denote the length of the code-word assigned to integer $N$ by WTC defined by (2). Let

[TABLE]

and

[TABLE]

Then, for all $N\geq 5$, we have

[TABLE]

These bounds allow us to determine the asymptotic behaviour of the length of WTC1 code-words.

[TABLE]

where

[TABLE]

The proofs of Lemma 1 and Theorem 1 are deferred to Appendix A. Complementing the comparisons in section IV, an interesting consequence of Theorem 1 is that there exists some integer $M$ such that $\forall N>M$, $|\text{WTC}(N)|>|\text{omega}(N)|$, although the precise $M$ is unknown and likely inconceivably large.

Theorem 1 can be used to demonstrate both the universality and asymptotic optimality of WTC, building on Elias [1]. To achieve this we first note that the Elias delta code [1], which has asymptotic code-length $\log_{2}N+2\log_{2}\log_{2}N+O(1)$, is both universal and asymptotically optimal. From Theorem 1 it is clear that code-words of WTC is asymptotically shorter than those of Elias delta code. This establishes the universality and asymptotic optimality of WTC.

VI-B Asymptotic code-length formulas for WTC

It is useful to have a simple expression for the code-word lengths of integers under WTC. The requirement for such lengths arises in inductive inference by minimum encoding. It is common to use Rissanen’s $\log^{*}$ code-word length formula to provide an approximate length for the statement of integer parameters. WTC provides an alternative coding scheme in such settings. Theorem 1 suggests the approximate code-word length formula for WTC as:

[TABLE]

where $c$ is a constant. Possible choices for $c$ are:

• •

$c=2$, based on the upper-bound on $\varepsilon_{N}$ in Theorem 1, which ensures $L(N;c)$ is non-decreasing;

• •

$c=-0.5$, based on the lower-bound on $\varepsilon_{N}$; or

• •

$c=0.75$, which is the average of the two error bounds.

The accuracy of both the bounds given by Lemma 1 and the asymptotic expression (7) is demonstrated in Fig. 8. The figure shows a close correspondence between the asymptotic expression (4) and the exact code-length (2), particularly as $N$ increases.

VII Conclusions

The Wallace tree code (WTC1) for positive integers $N\geq 1$ has shorter code-words than the Elias omega (and Fibonacci) codes for most integers upto at least $2.6855\times 10^{505}$. Code-word length increases in steps of two from time to time as $N$ increases. When using the code to store or transmit a sequence of integers, the effect of a bit error is localised. A formula for the approximate code-word length was derived in section VI-B.

We note that there is a second recursive version of the code, $\text{WTC}_{\text{r}}$. It has the same code-word lengths as WTC but is based on a non-lexicographical ordering of code-words: For code-words of length $2f+1,f>1$, consider all partitions of $2f$ into $j$ and $k$ such that $f=j+k$. Order code-words of the form ‘1’++$v$++$w$, where sub-code-words $|v|=j,|w|=k$ and $j+k=2f$, on $v$ and within that on $w$, recursively.

As discussed in section II-B1, the standard Elias omega code in effect uses a unary code (“0…” $\Rightarrow$ length section, “1…” $\Rightarrow$ final value section) to indicate the number ($\geq 1$) of sections in a code-word. This unary code can be replaced by another code for positive integers, even recursively, giving the omega* code which is more efficient for huge values.

Importantly, ordering the various codes discussed on increasing asymptotic efficiency gives: Fibonacci, Elias delta, Wallace WTC, Elias omega, omega2 (sec.II-B1), and omega* (all but Fibonacci are asymptotically optimal in the sense of Elias [1]).

Acknowledgment

The authors would like to thank the late Chris Wallace ($1933$–$2004$).

Appendix A Proof of Lemma 1

The basic approach that we use is to lower and upper-bound the function $f(N)$ with two new functions $\underline{f}(N)$ and $\overline{f}(N)$, respectively. To do this, we find lower and upper-bounds, say $c\underline{C}_{f}$ and $c\overline{C}_{f}$ that are continuous in $f$, and solve both $N=c\underline{C}_{f}$ and $N=c\overline{C}_{f}$ for $f$. We first derive the upper-bound $\overline{f}$. Our starting point is the following lower-bound on $cC_{f}$ established by Topley [10]:

[TABLE]

Setting the right-hand-side of (8) to $N$ and taking logarithms of both sides yields

[TABLE]

We wish to solve the above equation for $f$, but a closed form solution does not exist due to the troublesome logarithmic term. Instead, we can use the bounds

[TABLE]

$\forall f,f_{0}>1$, where the last step is the result of a first order Taylor series expansion of $-(3/2)\log f$ around the point $f_{0}$, the convexity of $-\log f$ ensuring that the inequality holds. Using this in (9) yields the following lower-bound for $cC_{f}$:

[TABLE]

Solving $c\underline{C}_{f}=\log N$ for $f$ yields

[TABLE]

The above lower-bound holds for any value of $f_{0}>1$, but by a judicious choice of $f_{0}$ it can be tightened. Ignoring terms of order $o(f)$ in (9) and solving for $f$ yields an initial guess at $\overline{f}$ of $f_{0}=\log N/\log 4$. Using this in $\overline{f}(N;f_{0})$ yields (3), which satisfies $\overline{f}\geq f(N)$ for all $N\geq 5$. Substituting $\overline{f}$ for $f(N)$ in (2) yields the upper-bound.

We now derive $\underline{f}$. We start with the following upper-bound on $cC_{f}$

[TABLE]

Setting the right-hand-side of (10) equal to $N$ and taking logarithms of both sides yields

[TABLE]

As before, solving (11) directly for $f$ is impossible due to the logarithmic term. Instead we note that we can use the bounds

[TABLE]

for $f>1$ and $f_{1}>1$, along with the fact that $\log 4-(1/2)\log 9\pi<0$ to derive the following upper-bound for $cC_{f}$:

[TABLE]

We now note due to the strictly increasing nature of $c\overline{C}_{f}$ that if $f_{1}>f(N)$ then the solution $\underline{f}$ of $\log c\overline{C}_{f}=\log N$ will satisfy $\underline{f}<f_{1}$. We therefore choose $f_{1}=\overline{f}$, which we previously established is an upper-bound to $f(N)$ for $N\geq 5$, and solve for $\underline{f}$, yielding

[TABLE]

which satisfies $\underline{f}<f(N)$ for all $N\geq 5$. Substituting $\underline{f}(N)$ for $f(N)$ in (2) completes the proof. $\Box$

Appendix B Proof of Theorem 1

Let

[TABLE]

and let $\varepsilon_{N}=L(N)-L_{A}(N)$. If we rewrite $\overline{f}$ as

[TABLE]

then a straightforward application of L’Hopital’s rule shows that

[TABLE]

which itself implies (5) if we note that $\varepsilon_{N}\leq 2\overline{f}+1-L_{A}(N)$ (by application of Lemma 1). Similarly, by tedious algebra we can show that

[TABLE]

which implies (6) as $\varepsilon_{N}\geq 2\underline{f}+1-L_{A}(N)$, completing the proof. $\Box$