The smallest grammar problem revisited

TL;DR

This paper refines bounds on approximation ratios for several grammar-based compression algorithms, closing previous gaps and improving lower bounds, thus advancing theoretical understanding of their efficiency.

Contribution

It establishes precise asymptotic bounds for LZ78, BISECTION, and RePair algorithms, improving and closing gaps in the theoretical analysis of their approximation ratios.

Findings

LZ78 approximation ratio is Θ((n/log n)^{2/3})

BISECTION approximation ratio is Θ(√(n/log n))

RePair lower bound improved to Ω(log n / log log n)

Abstract

In a seminal paper of Charikar et al. on the smallest grammar problem, the authors derive upper and lower bounds on the approximation ratios for several grammar-based compressors, but in all cases there is a gap between the lower and upper bound. Here the gaps for $\mathsf{LZ78}$ and $\mathsf{BISECTION}$ are closed by showing that the approximation ratio of $\mathsf{LZ78}$ is $\Theta( (n/\log n)^{2/3})$, whereas the approximation ratio of $\mathsf{BISECTION}$ is $\Theta(\sqrt{n/\log n})$. In addition, the lower bound for $\mathsf{RePair}$ is improved from $\Omega(\sqrt{\log n})$ to $\Omega(\log n/\log\log n)$. Finally, results of Arpe and Reischuk relating grammar-based compression for arbitrary alphabets and binary alphabets are improved.

Equations

$$g(k,n)=\max \{g(w)\mid w\in [1,k{]}^n\},$$

$$w_k=\left(\prod _{i=1}^k{a}_{k-i+1}^5\right)\prod _{i=1}^{k-1}\left(\prod _{j=i+2}^k{\left(a_j{a}_i\right)}^2\right)a_{i+1}{a}_i{a}_{i+1}$$

$${\alpha }_{\mathcal{C}}(k,n)=\max \{{\alpha }_{\mathcal{C}}(w)\mid w\in [1,k{]}^n\}=\max \{|\mathcal{C}(w)|/g(w)\mid w\in [1,k{]}^n\}$$

$${\alpha }_{\mathsf{BI}}(2,n)$$

$${\alpha }_{\mathsf{BI}}(n)$$

$$u_k=\left(\prod _{j=0}^{k-2}{\mathrm{bin}}_k(j)a^{m_k}\right){\mathrm{bin}}_k(k-1),$$

$$s_k={\left(u_k{a}^{m_k+1}\right)}^{m_k}{u}_k.$$

$$F_k=\{a^i{\mathrm{bin}}_k(j)a^{m_k-i}\mid 0\le j\le k-1,0\le i\le m_k\}.$$

$$g(u_k^{\prime })\in \mathcal{O}\left(\frac{k\log k}{\log (k\log k)}\right)=\mathcal{O}\left(\frac{k\log k}{\log k+\log \log k}\right)=\mathcal{O}(k).$$

$${\alpha }_{LZ78}(2,n)$$

$${\alpha }_{LZ78}(n)$$

$$s_{m,k}=a^{\frac{k(k+1)}{2}}{b}^{m(2m+1)}{u}_{m,k}{v}_{m,k}.$$

$$\bigcup _{j=1}^k\left(\{a^{k-j+1}{b}^{i+1},b^{2m-i}{a}^j\mid 0\le i\le m-1\}\cup \{a^{k-j+1}{b}^{m+1},a^k{b}^{m+1}{a}^j\}\right).$$

$$u_{m,k,j}$$

$$\{a^{k-j+1}{b}^{i+1},b^{2m-i}{a}^j\mid 0\le i\le m-1\}\cup \{a^{k-j+1}{b}^{m+1},a^k{b}^{m+1}{a}^j\},$$

$$u_{m,k,1}$$

$$\left\{a^i{b}^p{a}^j\mid 0\le i\le k-1,1\le j\le k,1\le p\le m\right\}.$$

$$s_{2,4}=a^{10}{b}^{10}\underset{u_{2,4}}{\underbrace{((a^4{b}^{5}a{)}^2(a^4{b}^3{)}^2{)}^4{a}^4}}\underset{v_{2,4}}{\underbrace{(b{a}^4{b}^2{a}^4{)}^{16}}}.$$

$$\begin{array}[]{llllll}a^{4}b&b^{4}a&a^{4}b^{2}&b^{3}a&a^{4}b^{3}&a^{4}b^{3}a\\ a^{3}b&b^{4}a^{2}&a^{3}b^{2}&b^{3}a^{2}&a^{3}b^{3}&a^{4}b^{3}a^{2}\\ a^{2}b&b^{4}a^{3}&a^{2}b^{2}&b^{3}a^{3}&a^{2}b^{3}&a^{4}b^{3}a^{3}\\ ab&b^{4}a^{4}&ab^{2}&b^{3}a^{4}&ab^{3}&a^{4}b^{3}a^{4}\end{array}$$

$$\begin{array}[]{ll}ba&a^{3}b^{2}a\\ a^{3}ba&a^{3}b^{2}a^{2}\\ a^{2}ba&a^{3}b^{2}a^{3}\\ aba&a^{3}b^{2}a^{4}\\ ba^{2}&a^{2}b^{2}a\\ a^{3}ba^{2}&a^{2}b^{2}a^{2}\\ a^{2}ba^{2}&a^{2}b^{2}a^{3}\\ aba^{2}&a^{2}b^{2}a^{4}\\ ba^{3}&ab^{2}a\\ a^{3}ba^{3}&ab^{2}a^{2}\\ a^{2}ba^{3}&ab^{2}a^{3}\\ aba^{3}&ab^{2}a^{4}\\ ba^{4}&b^{2}a\\ a^{3}ba^{4}&b^{2}a^{2}\\ a^{2}ba^{4}&b^{2}a^{3}\\ aba^{4}&b^{2}a^{4}\end{array}$$

$${\alpha }_{\mathsf{RePair}}(n)\in \Omega \left(\sqrt{\log n}\right)$$

$${\alpha }_{\mathsf{RePair}}(2,n)\in \mathcal{O}\left((n/\log n{)}^{2/3}\right)$$

$$w_k=h(B_{\lceil \log k\rceil }[1:k]).$$

$$s_k=\prod _{i=1}^{k-1}\left(a^{w_k[1:k+i]}b\right)a^{w_k}=a^{w_k[1:k+1]}b{a}^{w_k[1:k+2]}b\dots a^{w_k[1:2k-1]}b{a}^{w_k},$$

$$v_i=f_{k-2}(w_k[i+2])f_{k-3}(w_k[i+3])\dots f_1(w_k[k+i-1])f_0(w_k[k+i])$$

$$S\to \underset{a^{20}}{\underbrace{X_3{X}_3{X}_2}}b\underset{a^{41}}{\underbrace{{X_3}^{5}a}}b\underset{a^{82}}{\underbrace{{X_3}^{10}{X}_1}}b\underset{a^{165}}{\underbrace{{X_3}^{20}{X}_{2}a}},$$

$$k\cdot (\lceil (k-3)/2\rceil -2\lceil \log k\rceil -1)\in \Theta (k^2)$$

$$\phi (c_i)=a^{i}b(0\le i\le k-1).$$

$$\ell (s)$$

$$m(s)$$

$$r(s)$$

$$\psi (s)=\ell (s)m(s)r(s).$$

$$\psi (v_0){\ell }_1{A}_1^{\prime }{r}_1\psi (v_1){\ell }_2{A}_2^{\prime }{r}_2\cdots \psi (v_{n-1}){\ell }_n{A}_n^{\prime }{r}_n\psi (v_n).$$

Full text

Keywords. string compression, smallest grammar problem, approximation algorithm, LZ78, RePair

A short version of this paper appeared in the Proceedings of SPIRE 2016 [16].

This work has been supported by the DFG research project LO 748/10-1 (QUANT-KOMP)

1. Introduction

1.1. Grammar-based compression

The idea of grammar-based compression is based on the fact that in many cases a word $w$ can be succinctly represented by a context-free grammar that produces exactly $w$. Such a grammar is called a straight-line program (SLP for short) for $w$. For instance, $S\to cAABB$, $A\to aab$, $B\to CC$, $C\to cb$ is an SLP for the word $caabaabcbcbcbcb$. SLPs were introduced independently by various authors in different contexts [32, 39, 3, 10] and under different names. For instance, in [3, 10] the term word chains was used since SLPs generalize addition chains from numbers to words. Probably the best known example of a grammar-based compressor is the classical LZ78-compressor of Lempel and Ziv [39]. Indeed, it is straightforward to transform the LZ78-representation of a word $w$ into an SLP for $w$. Other well-known grammar-based compressors are BISECTION [22], SEQUITUR [30], and RePair [26], just to mention a few.

A central question asked from the very beginning in the area of grammar-based compression is how to measure the quality of an SLP, or, more broadly, the quality of the grammar-based compressor that computes an SLP for a given input word. One can distinguish two main approaches for such quality measures: (i) bit-based approaches, where one analyzes the bit length of a suitable binary encoding of an SLP and (ii) size-based approaches which measure the quality of an SLP by its size. The size of an SLP is defined as the sum of the lengths of all right-hand sides of the SLP (the SLP from the previous paragraph has size 12). Let us briefly survey the literature on these two approaches before we explain our main results in Section 1.4.

1.2. Bit-based approaches

It seems that the first attempt at evaluating a grammar-based compressor was done for LZ78 by Ziv and Lempel [39], who developed their own methodology of comparing (finite state) compressors: In essence, given a word $w$ define $L_{s}(w)$ as the length of an appropriate bit encoding of the output produced by LZ78 with window-size $s$ on input $w$ and by $L_{s}^{*}(w)$ the smallest bit-size achievable by a finite-state compressor with $s$ states on input $w$. It was shown that $\lim_{s\to\infty}\limsup_{n\to\infty}L_{s}(w)/L_{s}^{*}(w)=1$. In other words, LZ78 is optimal (up to lower order terms) among finite-state compressors. (Note that the actual statement is more general, as it allows $w$ to be compressed after some initially read prefix, i.e., we compare how $w$ in $uw$ is compressed by LZ78 and other compressors).

Later, a systematic evaluation of grammar-based compressors was done using the information theoretic paradigm. In [21, 22, 24, 37], grammar-based compressors have been used in order to construct universal codings in the following sense: for every finite state source and every input string $w$ of length $n$ (that is emitted with non-zero probability by the source), the coding length of $w$ is bounded by $-\log_{2}P(w)+R(n)$, where $P(w)$ is the probability that the source emits $w$ (thus $-\log_{2}P(w)$ is the self-information of $w$) and $R(n)$ is a function in $o(n)$. The function $R(n)/n$ is called the redundancy; it converges to zero. In [21, 22, 24, 37] the code for $w$ is constructed in two steps: First, an SLP is computed for $w$ using a grammar-based compressor. In a second step this SLP is encoded by a bit string using a suitable binary encoding (see also [36] for the problem of encoding SLPs within the information-theoretic limit). In [21] it was shown that the redundancy can be bounded by $\mathcal{O}(\log\log n/\log n)$ provided the grammar-based compressor produces an SLP of size $\mathcal{O}(n/\log n)$ for every input string of length $n$ (this assumes an alphabet of constant size; otherwise an additional factor $\log\sigma$ enters the bounds). The size bound $\mathcal{O}(n/\log n)$ holds for all grammar-based compressors that produce so-called irreducible SLPs [21], which roughly speaking means that certain redundancies in the SLP are eliminated. Moreover, every SLP can be easily made irreducible by a simple post-processing [21]. In [24], the redundancy bound from [21] was improved to $\mathcal{O}(1/\log n)$ for so-called structured grammar-based codes.

Recently, bounds in terms of the $k$-th order empirical entropy $H_{k}(w)$ of the input string $w$ have been shown for grammar-based compressors [13, 31]. Again, these results assume a suitable binary encoding of SLPs. In [31] it was shown that the length of the binary encoding (using the encoding from [21]) of an irreducible SLP for a string $w$ can be bounded by $H_{k}(w)\cdot|w|+\mathcal{O}(nk\log\sigma/\log_{\sigma}n)$, where $n$ is the length of the input string and $\sigma$ is the size of the alphabet. Note that the additional additive term $\mathcal{O}(nk\log\sigma/\log_{\sigma}n)$ is in $o(n\log\sigma)$ under the standard assumption that $k=o(\log_{\sigma}n)$. In [13] similar bounds are derived for more natural binary encodings of SLPs. On the other hand, a lower bound of $H_{k}(w)\cdot|w|+\Omega(nk\log\sigma/\log_{\sigma}n)$ was recently shown for a wide class of “natural” grammar-based compressors [14]. Hence, the mentioned upper bounds from [13, 31] are tight.

1.3. Size-based approaches

Bit-based approaches analyze the length of the binary encoding of the SLP. For this, one has to fix a concrete binary encoding. In contrast, the size of the SLP (the sum of the lengths of all right-hand sides) abstracts away from the concrete binary encoding of the SLP. Analyzing this more abstract quality measure has also some advantages: SLPs turned out to be particularly useful for the algorithmic processing of compressed data. For many algorithmic problems on strings, efficient algorithms are known in the setting where the input strings are represented by SLPs, see [5, 28] for some examples. For the running time of these algorithms, the size of the input SLPs is the main parameter whereas the concrete binary encoding of the SLPs is not relevant. Another research direction where only the SLP size is relevant arises from the recent work on string attractors, where the size of a smallest SLP for a string is compared with other string parameters that arise from dictionary compression (number of phrases in the LZ77 parse, minimal number of phrases in a bidirectional parse, number of runs in the Burrows-Wheeler transform) [20].

Another important aspect when comparing the bit-based approach (in particular, entropy bounds for binary encoded SLPs) and the size-based approach was also emphasized in [7, Section VI]: entropy bounds are often no longer useful when low-entropy strings are considered; see also [25] for an investigation in the context of Lempel-Ziv compression. Consider for instance the entropy bounds in [13, 31]. Besides the $k$-th order empirical entropy of the input string, these bounds also contain an additive term of order $\mathcal{O}(nk\log\sigma/\log_{\sigma}n)$ (and by the result from [14] this is unavoidable). Similar remarks apply to the redundancy bound in [21], where the output bit length of the grammar-based compressor is bounded by the self-information of the input string (with respect to a $k$-th order finite state source) plus a term of order $\mathcal{O}(n(k+\log\log_{\sigma}n)/\log_{\sigma}n)$. For input strings with low entropy/self-information these additive terms can be much larger than the entropy/self-information. For such input strings the existing entropy/redundancy bounds do not make useful statements about the performance of a grammar-based compressor.

A first investigation of the SLP size was done by Berstel and Brlek [3], who proved that the function $g(\sigma,n)=\max\{g(w)\mid w\in\{1,\ldots,\sigma\}^{n}\}$, where $g(w)$ is the size of a smallest SLP for the word $w$, is in $\Theta(n/\log_{\sigma}n)$. Note that $g(\sigma,n)$ measures the worst case SLP-compression over all words of length $n$ over an alphabet of size $\sigma$. It is worth noting that addition chains [38] are basically SLPs over a singleton alphabet and that $g(1,n)$ is the size of a smallest addition chain for $n$ (up to a constant factor).

Constructing a smallest SLP for a given input word is known as the smallest grammar problem. Storer and Szymanski [35] and Charikar et al. [7] proved that it cannot be solved in polynomial time unless $\mathsf{P}=\mathsf{NP}$. Moreover, Charikar et al. [7] showed that, unless $\mathsf{P}=\mathsf{NP}$, one cannot compute in polynomial time for a given word $w$ an SLP of size $<(8569/8568)\cdot g(w)$. The construction in [7] uses an alphabet of unbounded size, and it was unknown whether this lower bound holds also for words over a fixed alphabet. In [7] it is stated that the construction in [35] shows that the smallest grammar problem for words over a ternary alphabet cannot be solved in polynomial time unless $\mathsf{P}=\mathsf{NP}$. But this is not clear at all, see the recent paper [6] for a detailed explanation. In the same paper [6] it was shown that the smallest grammar problem for an alphabet of size 24 cannot be solved in polynomial time unless $\mathsf{P}=\mathsf{NP}$ using a rather complex construction. It is far from clear whether this construction can be adapted so that it works also for a binary alphabet. Another idea for showing $\mathsf{NP}$-hardness of the smallest grammar problem for binary words is to reduce the smallest grammar problem for unbounded alphabets to the smallest grammar problem for a binary alphabet. This route was investigated in [2], where the following result was shown for every constant $c$: If there is a polynomial time grammar-based compressor that computes an SLP of size $c\cdot g(w)$ for a given binary input word $w$, then for every $\varepsilon>0$ there is a polynomial time grammar-based compressor that computes an SLP of size $(24c+\varepsilon)\cdot g(w)$ for a given input word $w$ over an arbitrary alphabet. The construction in [2] uses a quite technical block encoding of arbitrary alphabets into a binary alphabet.

A size-based quality measure for grammar-based compressors is the approximation ratio [7]: For a given grammar-based compressor $\mathcal{C}$ that computes from a given word $w$ an SLP $\mathcal{C}(w)$ for $w$ one defines the approximation ratio of $\mathcal{C}$ on $w$ as the quotient of the size of $\mathcal{C}(w)$ and the size $g(w)$ of a smallest SLP for $w$. The approximation ratio $\alpha_{\mathcal{C}}(n)$ is the maximal approximation ratio of $\mathcal{C}$ among all words of length $n$ over any alphabet. The approximation ratio is a useful measure for the worst-case performance of a grammar-based compressor, where the worst-case over all strings of a certain length is considered. This includes also low-entropy strings, for which the existing entropy/redundancy bounds are no longer useful as argued above. In this context one should also emphasize the fact that the entropy/redundancy bounds from [21, 31] apply to all grammar-based compressors that produce irreducible SLPs. As mentioned above, this property can be easily enforced by a simple post-processing of the SLP. This shows that the entropy/redundancy bounds from [21, 31] are not useful for a fine-grained comparison of grammar-based compressors. In contrast, analyzing the approximation ratio can lead to such a fine-grained comparison.

Charikar et al. [7] initiated a systematic investigation of the approximation ratio of various grammar-based compressors (LZ78, BISECTION, Sequential, RePair, LongestMatch, Greedy). They proved lower and upper bounds for the approximation ratios of theses compressor, but for none of them the lower and upper bounds match. Moreover, Charikar et al. present a linear time grammar-based compressor with an approximation ratio of $\mathcal{O}(\log n)$. Other linear time grammar-based compressors which achieve the same approximation ratio can be found in [18, 19, 33, 34]. It is unknown whether there exist grammar-based compressors that work in polynomial time and have an approximation ratio of $o(\log n)$. Getting a polynomial time grammar-based compressor with an approximation ratio of $o(\log n/\log\log n)$ would solve a long-standing open problem on addition chains [7, 38].

1.4. Results of the paper

Our first main contribution (Section 3) is an improved analysis of the approximation ratios of LZ78, BISECTION, and RePair. These compression algorithms are among the most popular grammar-based compressors. LZ78 is a classical algorithm and the basis of several widely used text compressors such as LZW (Lempel-Ziv-Welch). RePair shows in many applications the best compression results among the tested grammar-based compressors [4, 12] and found applications, among others, in web graph compression [8], different scenarios related to word-based text compression [40], searching compressed text [41], suffix array compression [42] and (in a slightly modified form in) XML compression [27]. Some variants and improvements of RePair can be found in [4, 11, 12, 15, 29]. BISECTION was first studied in the context of universal lossless compression [22] (called MPM there). On bit strings of length $2^{n}$, BISECTION produces in fact the ordered binary decision diagram (OBDD) of the Boolean function represented by the bit string; see also [23]. OBDDs are a widely used data structure in the area of hardware verification.

For LZ78 and BISECTION we close the gaps for the approximation ratio from [7]. For this we improve the corresponding lower bounds from [7] and obtain the approximation ratios $\Theta((n/\log n)^{1/2})$ for BISECTION and $\Theta((n/\log n)^{2/3})$ for LZ78. We prove both lower bounds using a binary alphabet. These are the first exact (up to constant factors) approximation ratios for practical grammar-based compressors. We also improve the lower bound for RePair from $\Omega\left(\sqrt{\log n}\right)$ to $\Omega\left(\log n/\log\log n\right)$ using a binary alphabet. The previous lower bound from [7] used a family of words over an alphabet of unbounded size. Our new lower bound for RePair is still quite far away from the best known upper bound of $\mathcal{O}((n/\log n)^{2/3})$ [7]. On the other hand, the lower bound $\Omega\left(\log n/\log\log n\right)$ is of particular interest, since it was shown in [7] that a grammar-based compressor with an approximation ratio of $o(\log n/\log\log n)$ would improve Yao’s method for computing a smallest addition chain for a set of numbers [38], which is a long standing open problem. Our new lower bound excludes RePair as a candidate for improving Yao’s method. Let us also remark that RePair belongs to the class of so-called global grammar-based compressors (other examples are LongestMatch [21] and Greedy [1]). Analyzing the approximation ratio of global algorithms seems to be very difficult. We can quote here Charikar et al. [7]: “Because they [global algorithms] are so natural and our understanding is so incomplete, global algorithms are one of the most interesting topics related to the smallest grammar problem that deserve further investigation.” In the specific context of singleton alphabets, a detailed investigation of the approximation ratios of global grammar-based compressors was recently examined in [17].

Our second main contribution deals with the hardness of the smallest grammar problem for words over a binary alphabet. As mentioned above, it is open whether this problem is NP-hard. This is one of the most intriguing unsolved problems in the area of grammar-based compression. Recall that Arpe and Reischuk [2] used a quite technical block encoding to show that if there is a polynomial time grammar-based compressor with approximation ratio $c$ (a constant) on binary words, then there is a polynomial time grammar-based compressor with approximation ratio $24c+\varepsilon$ for every $\varepsilon>0$ on arbitrary words. Here, we present a very simple construction, which encodes the $i$-th alphabet symbol by $a^{i}b$, and yields the same result as in [2] but with $24c+\varepsilon$ replaced by $6c$. In order to show NP-hardness of the smallest grammar problem for binary strings, one would have to reduce the factor 6 to at most $8569/8568$. This follows from the inapproximability result for the smallest grammar problem from [7].

2. Straight-line Programs

Let $w=a_{1}\cdots a_{n}$ ($a_{1},\dots,a_{n}\in\Sigma$) be a word over an alphabet $\Sigma$. The length $|w|$ of $w$ is $n$ and we denote by $\varepsilon$ the word of length [math]. Let $\Sigma^{+}=\Sigma^{*}\setminus\{\varepsilon\}$ be the set of nonempty words. For $w\in\Sigma^{+}$, we call $v\in\Sigma^{+}$ a factor of $w$ if there exist $x,y\in\Sigma^{*}$ such that $w=xvy$. If $x=\varepsilon$ (respectively $y=\varepsilon$) then we call $v$ a prefix (respectively suffix) of $w$. A factorization of $w$ is a decomposition $w=f_{1}\cdots f_{\ell}$ into factors $f_{1},\dots,f_{\ell}$. For words $w_{1},\dots,w_{n}\in\Sigma^{*}$, we further denote by $\prod_{i=j}^{n}w_{i}$ the word $w_{j}w_{j+1}\cdots w_{n}$ if $j\leq n$ and $\varepsilon$ otherwise.

A straight-line program, briefly SLP, is a context-free grammar that produces a single word $w\in\Sigma^{+}$. Formally, it is a tuple $\mathbb{A}=(N,\Sigma,P,S)$, where $N$ is a finite set of nonterminals with $N\cap\Sigma=\emptyset$, $S\in N$ is the start nonterminal, and $P$ is a finite set of productions (or rules) of the form $A\to w$ for $A\in N$, $w\in(N\cup\Sigma)^{+}$ such that: (i) For every $A\in N$, there exists exactly one production of the form $A\to w$, and (ii) the binary relation $\{(A,B)\in N\times N\mid(A\to w)\in P,\;B\text{ occurs in }w\}$ is acyclic. Every nonterminal $A\in N$ produces a unique string $\mathrm{val}_{\mathbb{A}}(A)\in\Sigma^{+}$. The string defined by $\mathbb{A}$ is $\mathrm{val}(\mathbb{A})=\mathrm{val}_{\mathbb{A}}(S)$. We omit the subscript $\mathbb{A}$ when it is clear from the context. The size of the SLP $\mathbb{A}$ is $|\mathbb{A}|=\sum_{(A\to w)\in P}|w|$. We denote by $g(w)$ the size of a smallest SLP producing the word $w\in\Sigma^{+}$. It is easy to see that $g(w)\leq|w|$ since for each word $w$ there is a trivial SLP with the only rule $S\to w$. We will use the following lemma which summarizes known results about SLPs.

Lemma 2.1**.**

Let $\Sigma$ be a finite alphabet of size $\sigma$.

1. (1)

For every word $w\in\Sigma^{+}$ of length $n$, there exists an SLP $\mathbb{A}$ of size \mathcal{O}\big{(}\frac{n}{\log_{\sigma}n}\big{)} such that $\mathrm{val}(\mathbb{A})=w$. 2. (2)

For an SLP $\mathbb{A}$ and a number $n>0$, there exists an SLP $\mathbb{B}$ of size $|\mathbb{A}|+\mathcal{O}(\log n)$ such that $\mathrm{val}(\mathbb{B})=\mathrm{val}(\mathbb{A})^{n}$. 3. (3)

For SLPs $\mathbb{A}_{1}$ and $\mathbb{A}_{2}$ there exists an SLP $\mathbb{B}$ of size $|\mathbb{A}_{1}|+|\mathbb{A}_{2}|$ such that $\mathrm{val}(\mathbb{B})=\mathrm{val}(\mathbb{A}_{1})\mathrm{val}(\mathbb{A}_{2})$. 4. (4)

For given words $w_{1},\dots,w_{n}\in\Sigma^{*}$, $u\in\Sigma^{+}$ and SLPs $\mathbb{A}_{1},\mathbb{A}_{2}$ with $\mathrm{val}(\mathbb{A}_{1})=u$ and $\mathrm{val}(\mathbb{A}_{2})=w_{1}xw_{2}x\cdots w_{n-1}xw_{n}$ for a symbol $x\not\in\Sigma$, there exists an SLP $\mathbb{B}$ of size $|\mathbb{A}_{1}|+|\mathbb{A}_{2}|$ such that $\mathrm{val}(\mathbb{B})=w_{1}uw_{2}u\cdots w_{n-1}uw_{n}$. 5. (5)

A string $w\in\Sigma^{*}$ contains at most $g(w)\cdot k$ distinct factors of length $k$.

Statement 1 can be found for instance in [3]. Statements 2, 3 and 5 are shown in [7]. The proof of 4 is straightforward: Simply replace in the SLP $\mathbb{A}_{2}$ every occurrence of the terminal $x$ by the start nonterminal of $\mathbb{A}_{1}$ and add all rules of $\mathbb{A}_{1}$ to $\mathbb{A}_{2}$.

The maximal size of a smallest SLP for all words of length $n$ over an alphabet of size $k$ is

[TABLE]

where $[1,k]=\{1,\ldots,k\}$. By point 1 of Lemma 2.1 we have $g(k,n)\in\mathcal{O}(n/\log_{k}n)$. In fact, Berstel and Brlek proved in [3] that $g(k,n)\in\Theta(n/\log_{k}n)$. As a first minor result, we show that there are words of length $2k^{2}+2k+1$ over an alphabet of size $k$ for which the size of a smallest SLP equals the word length. Additionally, we show that all longer words have strictly smaller SLPs. Together this yield the following proposition:

Proposition 2.2**.**

Let $n_{k}=2k^{2}+2k+1$ for $k>0$. Then (i) $g(k,n)<n$ for $n>n_{k}$ and (ii) $g(k,n)=n$ for $n\leq n_{k}$.

Proof.

Let $\Sigma_{k}=\{a_{1},\ldots,a_{k}\}$ and let $M_{n,\ell}\subseteq\Sigma_{k}^{*}$ be the set of all words $w$ where a factor $v$ of length $\ell$ occurs at least $n$ times without overlap. It is easy to see that $g(w)<|w|$ if and only if $w\in M_{3,2}\cup M_{2,3}$. Hence, we have to show that every word $w\notin M_{3,2}\cup M_{2,3}$ has length at most $2k^{2}+2k+1$. Moreover, we present words $w_{k}\in{\Sigma_{k}}^{*}$ of length $2k^{2}+2k+1$ such that $w_{k}\notin M_{3,2}\cup M_{2,3}$.

Let $w\notin M_{3,2}\cup M_{2,3}$. Consider a factor $a_{i}a_{j}$ of length two. If $i\neq j$ then this factor does not overlap itself, and thus $a_{i}a_{j}$ occurs at most twice in $w$. Now consider $a_{i}a_{i}$. Then $w$ contains at most four (possibly overlapping) occurrence of $a_{i}a_{i}$, because five occurrences of $a_{i}a_{i}$ would yield at least three non-overlapping occurrences of $a_{i}a_{i}$. It follows that $w$ has at most $2(k^{2}-k)+4k$ positions where a factor of length $2$ starts, which implies $|w|\leq 2k^{2}+2k+1$.

Now we create a word $w_{k}\notin M_{3,2}\cup M_{2,3}$ which realizes the above maximal occurrences of factors of length $2$:

[TABLE]

For example we have $w_{3}=a_{3}^{5}a_{2}^{5}a_{1}^{5}(a_{3}a_{1})^{2}a_{2}a_{1}a_{2}a_{3}a_{2}a_{3}$. One can check that $|w_{k}|=2k^{2}+2k+1$ and $w_{k}\notin M_{3,2}\cup M_{2,3}$. ∎

3. Approximation ratio

As mentioned in the introduction, there is no polynomial time algorithm that computes a smallest SLP for a given word, unless $\mathsf{P}=\mathsf{NP}$ [7, 35]. This result motivates approximation algorithms which are called grammar-based compressors. A grammar-based compressor $\mathcal{C}$ computes for a word $w$ an SLP $\mathcal{C}(w)$ such that $\mathrm{val}(\mathcal{C}(w))=w$. The approximation ratio $\alpha_{\mathcal{C}}(w)$ of $\mathcal{C}$ for an input $w$ is defined as $|\mathcal{C}(w)|/g(w)$. The worst-case approximation ratio $\alpha_{\mathcal{C}}(k,n)$ of $\mathcal{C}$ is the maximal approximation ratio over all words of length $n$ over an alphabet of size $k$:

[TABLE]

In this definition, $k$ might depend on $n$. Of course we must have $k\leq n$ and we write $\alpha_{\mathcal{C}}(n)$ instead of $\alpha_{\mathcal{C}}(n,n)$. This corresponds to the case where there is no restriction on the alphabet at all and it is the definition of the worst-case approximation ratio in [7]. The grammar-based compressors studied in our work are BISECTION [22], LZ78 [39] and RePair [26]. We will abbreviate the approximation ratio of BISECTION by $\alpha_{\sf BI}{}$. The families of words which we will use to improve the lower bounds of $\alpha_{{\sf BI}{}}(n)$ and $\alpha_{{\sf LZ78}{}}(n)$ are inspired by the constructions in [7].

3.1. BISECTION

The BISECTION algorithm [22] first splits an input word $w$ with $|w|\geq 2$ as $w=w_{1}w_{2}$ such that $|w_{1}|=2^{j}$ for the unique number $j\geq 0$ with $2^{j}<|w|\leq 2^{j+1}$. This process is recursively repeated with $w_{1}$ and $w_{2}$ until we obtain words of length $1$. During the process, we introduce a nonterminal for each distinct factor of length at least two and create a rule with two symbols on the right-hand side corresponding to the split. Note that if $w=u_{1}u_{2}\cdots u_{k}$ with $|u_{i}|=2^{n}$ for all $i,1\leq i\leq k$, then the SLP produced by BISECTION contains a nonterminal for each distinct word $u_{i}$ ($1\leq i\leq k$).

Example 3.1**.**

BISECTION* constructs an SLP for $w=ababbbaabbaaab$ as follows:*

• •

$w=w_{1}w_{2}$* with $w_{1}=ababbbaa$, *$w_{2}=bbaaab$

Introduced rule: $S\to W_{1}W_{2}$

• •

$w_{1}=x_{1}x_{2}$* with $x_{1}=abab$, $x_{2}=bbaa$, and $w_{2}=x_{2}x_{3}$ with *$x_{3}=ab$

Introduced rules: $W_{1}\to X_{1}X_{2},\;W_{2}\to X_{2}X_{3}$, $X_{3}\to ab$

• •

$x_{1}=x_{3}x_{3}$*, $x_{2}=y_{1}y_{2}$ with $y_{1}=bb$ and $y_{2}=aa$ *

Introduced rules: $X_{1}\to X_{3}X_{3},\;X_{2}\to Y_{1}Y_{2},\;Y_{1}\to bb,\;Y_{2}\to aa$

BISECTION performs asymptotically optimal on unary words $a^{n}$ since it produces an SLP of size $\mathcal{O}(\log n)$. Therefore $\alpha_{\sf BI}{}(1,n)\in\Theta(1)$. The following bounds on the approximation ratio for alphabets of size at least two are proven in [7, Thm. 5 and 6]:

[TABLE]

We improve the lower bound (1) so that it matches the upper bound (2):

Theorem 3.2**.**

For every $k,2\leq k\leq n$ we have $\alpha_{\sf BI}{}(k,n)\in\Theta(\sqrt{n/\log n})$.

Proof.

The upper bound (2) implies that $\alpha_{\sf BI}{}(k,n)\in\mathcal{O}(\sqrt{n/\log n})$ for all $k,2\leq k\leq n$. So it suffices to show $\alpha_{\sf BI}{}(2,n)\in\Omega(\sqrt{n/\log n})$. We first show that $\alpha_{\sf BI}{}(3,n)\in\Omega(\sqrt{n/\log n})$. In a second step, we encode a ternary alphabet into a binary alphabet while preserving the approximation ratio.

For every $k\geq 2$ let $\mathrm{bin}_{k}:\{0,1,\dots,k-1\}\to\{0,1\}^{\lceil\log_{2}k\rceil}$ be the function where $\mathrm{bin}_{k}(j)$ ($0\leq j\leq k-1$) is the binary representation of $j$ padded with leading zeros (e.g. $\mathrm{bin}_{9}(3)=0011$). We further define for every $k\geq 2$ the word

[TABLE]

where $m_{k}=2^{k-\lceil\log_{2}k\rceil}-\lceil\log_{2}k\rceil$. For instance $k=4$ leads to $m_{k}=2$ and $u_{4}=00aa01aa10aa11$. We analyze the approximation ratio $\alpha_{\sf BI}{}(s_{k})$ for the word

[TABLE]

Claim 1**.**

The SLP produced by BISECTION on input $s_{k}$ has size $\Omega(2^{k})$.

Proof*.*

If $s_{k}$ is split into non-overlapping factors of length $m_{k}+\lceil\log_{2}k\rceil=2^{k-\lceil\log_{2}k\rceil}$, then the resulting set $F_{k}$ of factors is

[TABLE]

For example $s_{4}$ consecutively consists of the factors $00aa$, $01aa$, $10aa$, $11aa$, $a00a$, $a01a$, $a10a$, $a11a$, $aa00$, $aa01$, $aa10$ and $aa11$. The size of $F_{k}$ is $(m_{k}+1)\cdot k\in\Theta(2^{k})$, because all factors are pairwise different and $m_{k}\in\Theta(2^{k}/k)$. It follows that the SLP produced by BISECTION on input $s_{k}$ has size $\Omega(2^{k})$, because the length of each factor in $F_{k}$ is a power of two and thus BISECTION creates a nonterminal for each distinct factor in $F_{k}$. (end proof of Claim 1)

Claim 2**.**

A smallest SLP producing $s_{k}$ has size $\mathcal{O}(k)$.

Proof*.*

There is an SLP of size $\mathcal{O}(\log m_{k})=\mathcal{O}(k)$ for the word $a^{m_{k}}$ by Lemma 2.1 (point 2). This yields an SLP for $u_{k}$ of size $\mathcal{O}(k)+g(u_{k}^{\prime})$ by Lemma 2.1 (point 4), where $u_{k}^{\prime}=(\prod_{i=0}^{k-2}\mathrm{bin}_{k}(i)x)\mathrm{bin}_{k}(k-1)$ is obtained from $u_{k}$ by replacing all occurrences of $a^{m_{k}}$ by a fresh symbol $x$. The word $u_{k}^{\prime}$ has length $\Theta(k\log k)$. Applying point 1 of Lemma 2.1 (note that $u^{\prime}_{k}$ is a word over a ternary alphabet) it follows that

[TABLE]

Hence $g(u_{k})\in\mathcal{O}(k)$. Finally, the SLP of size $\mathcal{O}(k)$ for $u_{k}$ yields an SLP of size $\mathcal{O}(k)$ for $s_{k}$ again using Lemma 2.1 (points 2 and 3). (end proof of Claim 2)

In conclusion: We showed that a smallest SLP for $s_{k}$ has size $\mathcal{O}(k)$, while BISECTION produces an SLP of size $\Omega(2^{k})$.This implies $\alpha_{\sf BI}{}(s_{k})\in\Omega(2^{k}/k)$. Let $n=|s_{k}|$. Since $s_{k}$ is the concatenation of $\Theta(2^{k})$ factors of length $\Theta(2^{k}/k)$, we have $n\in\Theta(2^{2k}/k)$ and thus $\sqrt{n}\in\Theta(2^{k}/\sqrt{k})$. This yields $\alpha_{\sf BI}{}(s_{k})\in\Omega(\sqrt{n/k})$. Together with $k\in\Theta(\log n)$ we obtain $\alpha_{\sf BI}{}(3,n)\in\Omega(\sqrt{n/\log n})$.

Let us now encode words over $\{0,1,a\}$ into words over $\{0,1\}$. Consider the homomorphism $f:\{0,1,a\}^{*}\to\{0,1\}^{*}$ with $f(0)=00$, $f(1)=01$ and $f(a)=10$. Then we can prove the same approximation ratio of BISECTION for the input $f(s_{k})\in\{0,1\}^{*}$ that we proved for $s_{k}$ above: The size of a smallest SLP for $f(s_{k})$ is at most twice as large as the size of a smallest SLP for $s_{k}$, because an SLP for $s_{k}$ can be transformed into an SLP for $f(s_{k})$ by replacing every occurrence of a symbol $x\in\{0,1,a\}$ by $f(x)$. Moreover, if we split $f(s_{k})$ into non-overlapping factors of twice the length as we considered for $s_{k}$, then we obtain the factors from $f(F_{k})$, whose length is again a power of two. Since $f$ is injective, we have $|f(F_{k})|=|F_{k}|\in\Theta(2^{k})$. ∎

3.2. LZ78

The LZ78 algorithm on input $w\in\Sigma^{+}$ implicitly creates a list of words $f_{1},\dots,f_{\ell}$ (which we call the LZ78-factorization) with $w=f_{1}\cdots f_{\ell}$ such that the following properties hold, where we set $f_{0}=\varepsilon$:

• •

$f_{i}\neq f_{j}$ for all $i,j,0\leq i,j\leq\ell-1$, with $i\neq j$.

• •

For all $i,1\leq i\leq\ell-1$, there exist $j,0\leq j<i$ and $a\in\Sigma$ such that $f_{i}=f_{j}a$.

• •

$f_{\ell}=f_{i}$ for some $0\leq i\leq\ell-1$.

Note that the LZ78-factorization is unique for each word $w$. To compute it, the ${\sf LZ78}{}$ algorithm needs $\ell$ steps performed by a single left-to-right pass. In the $k^{\text{th}}$ step ($1\leq k\leq\ell-1$) it chooses the factor $f_{k}$ as the shortest prefix of the unprocessed suffix $f_{k}\cdots f_{\ell}$ such that $f_{k}\neq f_{i}$ for all $i<k$. If there is no such prefix, then the end of $w$ is reached and the algorithm sets $f_{\ell}$ to the (possibly empty) unprocessed suffix of $w$.

The factorization $f_{1},\dots,f_{\ell}$ yields an SLP for $w$ of size at most $3\ell$ as described in the following example:

Example 3.3**.**

The LZ78-factorization of $w=aabaaababababaa$ is $a$, $ab$, $aa$, $aba$, $b$, $abab$, $aa$ and leads to an SLP with the following rules:

• •

$S\to F_{1}F_{2}F_{3}F_{4}F_{5}F_{6}F_{3}$**

• •

$F_{1}\to a,\;F_{2}\to F_{1}b,\;F_{3}\to F_{1}a,\;F_{4}\to F_{2}a,\;F_{5}\to b,\;F_{6}\to F_{4}b$**

We have a nonterminal $F_{i}$ for each factor $f_{i}$ ($1\leq i\leq 6$) such that $\mathrm{val}_{\mathbb{A}}(F_{i})=f_{i}$. The last factor $aa$ is represented in the start rule by the nonterminal $F_{3}$.

The LZ78-factorization of $a^{n}$ ($n>0$) is $a^{1},a^{2},\dots,a^{m},a^{k}$, where $k\in\{0,\dots,m\}$ such that $n=k+\sum_{i=1}^{m}i$. Note that $m\in\Theta(\sqrt{n})$ and thus $\alpha_{\sf LZ78}{}(1,n)\in\Theta(\sqrt{n}/\log n)$. The following bounds for the worst-case approximation ratio of LZ78 were shown in [7, Thm. 3 and 4]:

[TABLE]

We will improve the lower bound so that it matches the upper bound in (4).

Theorem 3.4**.**

For every $k,2\leq k\leq n$ we have $\alpha_{\sf LZ78}{}(k,n)\in\Theta((n/\log n)^{2/3})$.

Proof.

Due to (4) it suffices to show $\alpha_{\sf LZ78}{}(2,n)\in\Omega((n/\log n)^{2/3})$. For $k\geq 2,m\geq 1$, let $u_{m,k}=((a^{k}b^{(2m+1)}a)^{m}(a^{k}b^{(m+1)})^{2})^{k}a^{k}$ and $v_{m,k}=\left(\prod_{i=1}^{m}b^{i}a^{k}\right)^{k^{2}}$. We now analyze the approximation ratio of LZ78 on the words

[TABLE]

For example we have $u_{2,4}=((a^{4}b^{5}a)^{2}(a^{4}b^{3})^{2})^{4}a^{4}$, $v_{2,4}=(ba^{4}b^{2}a^{4})^{16}$ and $s_{2,4}=a^{10}\;b^{10}\;u_{2,4}\;v_{2,4}$.

Claim 1**.**

The SLP produced by ${\sf LZ78}{}$ on input $s_{m,k}$ has size $\Theta(k^{2}m)$.

Proof*.*

We consider the LZ78-factorization $f_{1},\dots,f_{\ell}$ of $s_{m,k}$. Example 3.5 gives a complete example. The prefix $a^{k(k+1)/2}$ produces the factors $f_{i}=a^{i}$ for every $i,1\leq i\leq k$ and the substring $b^{m(2m+1)}$ produces the factors $f_{k+i}=b^{i}$ for every $i,1\leq i\leq 2m$.

We next show that the substring $u_{m,k}$ then produces all factors from

[TABLE]

Let

[TABLE]

Then, $u_{m,k}=u_{m,k,1}\cdots u_{m,k,k}$. We show that each $u_{m,k,j}$ produces the factors from

[TABLE]

for each $j,1\leq j\leq k$, thus obtaining (5).

Consider

[TABLE]

From the factorization of the prefix $a^{\frac{k(k+1)}{2}}b^{m(2m+1)}$ of $s_{m,k}$, the first $a^{k}b^{2m+1}a$ is factorized into $a^{k}b$ and $b^{2m}a$. Next, for each of the following $a^{k}b^{2m+1}a$, we can see that the new factors are $a^{k}b^{2}$ and $b^{2m-1}a$, $a^{k}b^{3}$ and $b^{2m-2}a,\ldots,a^{k}b^{m}$ and $b^{m+1}a$. Finally, the remaining $a^{k}b^{m+1}a^{k}b^{m+1}a$ is factorized to $a^{k}b^{m+1}$ and $a^{k}b^{m+1}a$. Therefore, (6) gives the factors of $u_{m,k,j}$ for $j=1$.

Next, suppose that $u_{m,k,j^{\prime}}$ produces the factors shown in (6) for all $1<j^{\prime}<j$, and consider $u_{m,k,j}$. By the induction hypothesis, we see that $a^{k-j+1}b$ and $b^{2m}a^{j}$ are the first two factors. Similarly, we see that each of the following $a^{k-j+1}b^{2m+1}a^{j}$ is factorized to $a^{k-j+1}b^{2}$ and $b^{2m-1}a^{j}$, $a^{k-j+1}b^{3}$ and $b^{2m-2}a^{j},\ldots,a^{k-j+1}b^{m}$ and $b^{m+1}a^{j}$. Finally, the remaining suffix $a^{k-j+1}b^{m+1}a^{k}b^{m+1}a^{j}$ is factorized to $a^{k-j+1}b^{m+1}$ and $a^{k}b^{m+1}a^{j}$. It follows that the factorization of $u_{m,k}$ yields the factors shown in (5).

Next, we will show that the remaining suffix $v_{m,k}$ of $s_{m,k}$ produces the set of factors

[TABLE]

Observe that from the factors produced so far, only the factors $a^{j}b^{i}$ for $0\leq j\leq k,0\leq i\leq m$ can be used for the factorization of $v_{m,k}$. The reason for this is that all other factors contain an occurrence of $b^{m+1}$, which does not occur in $v_{m,k}$.

Let $x=k+2m+k(2m+2)$ and note that this is the number of factors that we have produced so far. The factorization of $v_{m,k}$ in $s_{m,k}$ slightly differs when $m$ is even, resp., odd. We now assume that $m$ is even and explain the difference to the other case afterwards. The first factor of $v_{m,k}$ in $s_{m,k}$ is $f_{x+1}=ba$. We already have produced the factors $a^{k-1}b^{i}$ for every $i,1\leq i\leq m$, and hence $f_{x+i}=a^{k-1}b^{i}a$ for every $i,2\leq i\leq m$ and $f_{x+m+1}=a^{k-1}ba$. The next $m$ factors are $f_{x+m+i}=a^{k-1}b^{i}a^{2}$ if $i$ is even, $f_{x+m+i}=a^{k-2}b^{i}a$ if $i$ is odd ($2\leq i\leq m$) and $f_{x+2m+1}=a^{k-2}ba$. This pattern continues: The next $m$ factors are $f_{x+2m+i}=a^{k-1}b^{i}a^{3}$ if $i$ is even, $f_{x+2m+i}=a^{k-3}b^{i}a$ if $i$ is odd ($2\leq i\leq m$) and $f_{x+3m+1}=a^{k-3}ba$ and so on. Hence, we get the following sets of factors for $(\prod_{i=1}^{m}b^{i}a^{k})^{k}$:

1. (i)

$\{a^{k-i}b^{p}a\mid 1\leq i\leq k,\;1\leq p\leq m,\;p\text{ is odd}\}$ for $f_{x+1},f_{x+3}\dots,f_{x+km-1}$ 2. (ii)

$\{a^{k-1}b^{p}a^{j}\mid 1\leq j\leq k,\;1\leq p\leq m,\;p\text{ is even}\}$ for $f_{x+2},f_{x+4},\dots,f_{x+km}$

The remaining word then starts with the factor $f_{y+1}=ba^{2}$, where $y=x+km$. Now the former pattern can be adapted to the next $k$ repetitions of $\prod_{i=1}^{m}b^{i}a^{k}$ which gives us

1. (i)

$\{a^{k-i}b^{p}a^{2}\mid 1\leq i\leq k,\;1\leq p\leq m,\;p\text{ is odd}\}$ for $f_{y+1},f_{y+3}\dots,f_{y+km-1}$ 2. (ii)

$\{a^{k-2}b^{p}a^{j}\mid 1\leq j\leq k,\;1\leq p\leq m,\;p\text{ is even}\}$ for $f_{y+2},f_{y+4},\dots,f_{y+km}$

The iteration of this process then reveals the whole pattern and thus yields the claimed factorization of $v_{m,k}$ in $s_{m,k}$ into factors $a^{i}b^{p}a^{j}$ for every $i,0\leq i\leq k-1$, $j,1\leq j\leq k$ and $p,1\leq p\leq m$. If $m$ is odd then the patterns in (i) and (ii) switch after each occurrence of $\prod_{i=1}^{m}b^{i}a^{k}$, which does not affect the result but makes the pattern slightly more complicated. But the case that $m$ is even suffices in order to derive the lower bound from the theorem.

We conclude that there are exactly $k+2m+k(2m+2)+k^{2}m$ factors (ignoring $f_{\ell}=\varepsilon$) and hence the SLP produced by ${\sf LZ78}{}$ on input $s_{m,k}$ has size $\Theta(k^{2}m)$. (end proof of Claim 1)

Claim 2**.**

A smallest SLP producing $s_{m,k}$ has size $\mathcal{O}(\log k+m)$.

Proof*.*

We will combine the points stated in Lemma 2.1 to prove this claim. Points 2 and 3 yield an SLP of size $\mathcal{O}(\log k+\log m)$ for the prefix $a^{k(k+1)/2}\;b^{m(2m+1)}\;u_{m,k}$ of $s_{m,k}$. To bound the size of an SLP for $v_{m,k}$ note at first that there is an SLP of size $\mathcal{O}(\log k)$ producing $a^{k}$ by point 2 of Lemma 2.1. Applying point 4 and again point 2, it follows that there is an SLP of size $\mathcal{O}(\log k)+g(v_{m,k}^{\prime})$ producing $v_{m,k}$, where $v_{m,k}^{\prime}=\prod_{i=1}^{m}b^{i}x$ for some fresh letter $x$. To get a small SLP for $v_{m,k}^{\prime}$, we can introduce $m$ nonterminals $B_{1},\dots,B_{m}$ producing $b^{1},\dots,b^{m}$ by adding rules $B_{1}\to b$ and $B_{i+1}\to B_{i}b$ ($1\leq i\leq m-1$). This is enough to get an SLP of size $\mathcal{O}(m)$ for $v_{m,k}^{\prime}$ and therefore an SLP of size $\mathcal{O}(\log k+m)$ for $v_{m,k}$. Together with our first observation and point 3 of Lemma 2.1 this yields an SLP of size $\mathcal{O}(\log k+m)$ for $s_{m,k}$. (end proof of Claim 2)

Claims 1 and 2 imply $\alpha_{\sf LZ78}{}(s_{m,k})\in\Omega(k^{2}m/(\log k+m))$. Let us now fix $m=\lceil\log k\rceil$. We get $\alpha_{\sf LZ78}{}(s_{m,k})\in\Omega(k^{2})$. Moreover, for the length $n=|s_{m,k}|$ of $s_{m,k}$ we have $n\in\Theta(k^{3}m+k^{2}m^{2})=\Theta(k^{3}\log k)$. We get $\alpha_{\sf LZ78}{}(s_{m,k})\in\Omega((n/\log k)^{2/3})$ which together with $\log n\in\Theta(\log k)$ finishes the proof. ∎

Example 3.5**.**

Here is the complete LZ78 factorization of

[TABLE]

Factors of $a^{10}$: $\;a,\;a^{2},\;a^{3},\;a^{4}$

Factors of $b^{10}$: $\;b,\;b^{2},\;b^{3},\;b^{4}$

Factors of $u_{2,4}$:

[TABLE]

Factors of $v_{2,4}$:

[TABLE]

3.3. RePair

For a given SLP $\mathbb{A}=(N,\Sigma,P,S)$, a word $\gamma\in(N\cup\Sigma)^{+}$ is called a maximal string of $\mathbb{A}$ if

• •

$|\gamma|\geq 2$,

• •

$\gamma$ appears at least twice without overlap in the right-hand sides of $\mathbb{A}$,

• •

and no strictly longer word appears at least as many times on the right-hand sides of $\mathbb{A}$ without overlap.

A global grammar-based compressor starts on input $w$ with the trivial SLP $\mathbb{A}=(\{S\},\Sigma,\{S\to w\},S)$. In each round, the algorithm selects a maximal string $\gamma$ of $\mathbb{A}$ and updates $\mathbb{A}$ by replacing a largest set of pairwise non-overlapping occurrences of $\gamma$ in $\mathbb{A}$ by a fresh nonterminal $X$. Additionally, the algorithm introduces the rule $X\to\gamma$. The algorithm stops when no maximal string occurs. The global grammar-based compressor RePair [26] selects in each round a most frequent maximal string. Note that the replacement is not unique, e.g. the word $a^{5}$ with the maximal string $\gamma=aa$ yields SLPs with rules $S\to XXa,X\to aa$ or $S\to XaX,X\to aa$ or $S\to aXX,X\to aa$. We assume the first variant in this paper, i.e. maximal strings are replaced from left to right.

The above description of RePair is taken from [7]. In most papers on RePair the algorithm works slightly different: It replaces in each step a digram (a string of length two) with the maximal number of pairwise non-overlapping occurrences in the right-hand sides. For example, for the string $w=abcabc$ this produces the SLP $S\to BB$, $B\to Ac$, $A\to ab$, whereas the RePair-variant from [7] produces the smaller SLP $S\to AA$, $A\to abc$.

The following lower and upper bounds on the approximation ratio of RePair were shown in [7]:

[TABLE]

The proof of the lower bound (8) assumes an alphabet of unbounded size. To be more accurate, the authors construct for every $k$ a word $w_{k}$ of length $\Theta(\sqrt{k}2^{k})$ over an alphabet of size $\Theta(k)$ such that $g(w)\in\mathcal{O}(k)$ and RePair produces a grammar of size $\Omega(k^{3/2})$ for $w_{k}$. We will improve this lower bound using only a binary alphabet. To do so, we first need to know how RePair compresses unary words.

Example 3.6** (unary inputs).**

RePair* produces on input $a^{27}$ the SLP with rules $X_{1}\to aa$, $X_{2}\to X_{1}X_{1}$, $X_{3}\to X_{2}X_{2}$ and $S\to X_{3}X_{3}X_{3}X_{1}a$, where $S$ is the start nonterminal. For the input $a^{22}$ only the start rule $S\to X_{3}X_{3}X_{2}X_{1}$ is different.*

In general, RePair creates on unary input $a^{m}$ ($m\geq 4$) the rules $X_{1}\to aa$, $X_{i}\to X_{i-1}X_{i-1}$ for $2\leq i\leq\lfloor\log m\rfloor-1$ and a start rule, which is strongly related to the binary representation of $m$ since each nonterminal $X_{i}$ produces the word $a^{2^{i}}$. To be more accurate, let $b_{\lfloor\log m\rfloor}b_{\lfloor\log m\rfloor-1}\cdots b_{1}b_{0}$ be the binary representation of $m$ and define the mappings $f_{i}$ ($i\geq 0$) by:

• •

$f_{0}:\{0,1\}\to\{a,\varepsilon\}$ with $f_{0}(1)=a$ and $f_{0}(0)=\varepsilon$,

• •

$f_{i}:\{0,1\}\to\{X_{i},\varepsilon\}$ with $f_{i}(1)=X_{i}$ and $f_{i}(0)=\varepsilon$ for $i\geq 1$.

Then the start rule produced by RePair on input $a^{m}$ is

$S\to X_{\lfloor\log m\rfloor-1}X_{\lfloor\log m\rfloor-1}f_{\lfloor\log m\rfloor-1}(b_{\lfloor\log m\rfloor-1})\cdots f_{1}(b_{1})f_{0}(b_{0})$.

This means that the symbol $a$ only occurs in the start rule if $b_{0}=1$, and the nonterminal $X_{i}$ ($1\leq i\leq\lfloor\log m\rfloor-2$) occurs in the start rule if and only if $b_{i}=1$. Since RePair only replaces words with at least two occurrences, the most significant bit $b_{\lfloor\log m\rfloor}=1$ is represented by $X_{\lfloor\log m\rfloor-1}X_{\lfloor\log m\rfloor-1}$. Note that for $1\leq m\leq 3$, RePair produces the trivial SLP $S\to a^{m}$.

For the proof of the new lower bound, we use De Bruijn sequences [9]. A binary De Bruijn sequence of order $n$ is a string $B_{n}\in\{0,1\}^{*}$ of length $2^{n}$ such that every string from $\{0,1\}^{n}$ is either a factor of $B_{n}$ or a suffix of $B_{n}$ concatenated with a prefix of $B_{n}$. Moreover, every word of length at least $n$ occurs at most once as factor in $B_{n}$. As an example, the string $1100$ is a De Bruijn sequence of order $2$, since $11$, $10$ and $00$ occur as factors and $01$ occurs as a suffix concatenated with a prefix. Lemma 2.1 (point 5) implies that every SLP for $B_{n}$ has size $\Omega(2^{n}/n)$.

Theorem 3.7**.**

$\alpha_{\mathsf{RePair}}(2,n)\in\Omega\left(\log n/\log\log n\right)$**

Proof.

We start with a binary De Bruijn sequence $B_{\lceil\log k\rceil}\in\{0,1\}^{*}$ of length $2^{\lceil\log k\rceil}$. We have $k\leq|B_{\lceil\log k\rceil}|<2k$. Since De Bruijn sequences are not unique, we fix a De Bruijn sequence which starts with $1$ for the remaining proof. We define a homomorphism $h:\{0,1\}^{*}\to\{0,1\}^{*}$ by $h(0)=01$ and $h(1)=10$. The words $w_{k}$ of length $2k$ are defined as

[TABLE]

For example $k=4$ and $B_{2}=1100$ yield $w_{4}=10100101$. We will analyze the approximation ratio of RePair for the binary words

[TABLE]

where the prefixes $w_{k}[1:k+i]$ for $1\leq i\leq k$ are interpreted as integers given by their binary representations. For example we have $s_{4}=a^{20}ba^{41}ba^{82}ba^{165}$.

Since $B_{\lceil\log k\rceil}[1]=w_{k}[1]=1$, we have $2^{k+i-1}\leq\left|a^{w_{k}[1:k+i]}\right|\leq 2^{k+i}-1$ for $1\leq i\leq k$ and thus $|s_{k}|\in\Theta\left(4^{k}\right)$.

Claim 1**.**

A smallest SLP producing $s_{k}$ has size $\mathcal{O}(k)$.

Proof*.*

There is an SLP $\mathbb{A}$ of size $\mathcal{O}(k)$ for the first $a$-block $a^{w_{k}[1:k+1]}$ of length $\Theta(2^{k})$. Let $A$ be the start nonterminal of $\mathbb{A}$. For the second $a$-block $a^{w_{k}[1:k+2]}$ we only need one additional rule: If $w_{k}[k+2]=0$, then we can produce $a^{w_{k}[1:k+2]}$ by the fresh nonterminal $B$ using the rule $B\to AA$. Otherwise, if $w_{k}[k+2]=1$, then we use $B\to AAa$. The iteration of that process yields for each $a$-block only one additional rule of size at most $3$. If we replace the $a$-blocks in $s_{k}$ by nonterminals as described, then the resulting word has size $2k+1$ and hence $g(s_{k})\in\mathcal{O}(k)$. (end proof of Claim 1)

Claim 2**.**

The SLP produced by RePair on input $s_{k}$ has size $\Omega(k^{2}/\log k)$.

Proof*.*

On unary inputs of length $m$, the start rule produced by RePair is strongly related to the binary encoding of $m$ as described above. On input $s_{k}$, the algorithm begins to produce a start rule which is similarly related to the binary words $w_{k}[1:k+i]$ for $1\leq i\leq k$. Consider the SLP $\mathbb{G}$ which is produced by RePair after $(k-1)$ rounds on input $s_{k}$. We claim that up to this point RePair is not affected by the $b$’s in $s_{k}$ and therefore has introduced the rules $X_{1}\to aa$ and $X_{i}\to X_{i-1}X_{i-1}$ for $2\leq i\leq k-1$. If this is true, then the first $a$-block is modified in the start rule after $k-1$ rounds as follows

$S\to X_{k-1}X_{k-1}f_{k-1}(w_{k}[2])f_{k-2}(w_{k}[3])\cdots f_{0}(w_{k}[k+1])b\cdots$

where $f_{0}(1)=a$, $f_{0}(0)=\varepsilon$ and $f_{i}(1)=X_{i}$, $f_{i}(0)=\varepsilon$ for $i\geq 1$. All other $a$-blocks are longer than the first one, hence each factor of the start rule which corresponds to an $a$-block begins with $X_{k-1}X_{k-1}$. Therefore, the number of occurrences of $X_{k-1}X_{k-1}$ in the SLP is at least $k$. Since the symbol $b$ occurs only $k-1$ times in $s_{k}$, it follows that our assumption is correct and RePair is not affected by the $b$’s in the first $(k-1)$ rounds on input $s_{k}$. Also, for each block $a^{w_{k}[1:k+i]}$, the $k-1$ least significant bits of $w_{k}[1:k+i]$ ($1\leq i\leq k$) are represented in the corresponding factor of the start rule of $\mathbb{G}$, i.e., the start rule contains non-overlapping factors $v_{i}$ with

[TABLE]

for $1\leq i\leq k$. For example after $3$ rounds on input $s_{4}=a^{20}ba^{41}ba^{82}ba^{165}$, we have the start rule

[TABLE]

where $v_{1}=X_{2}$, $v_{2}=a$, $v_{3}=X_{1}$ and $v_{4}=X_{2}a$. The length of the factor $v_{i}\in\{a,X_{1},\dots,X_{k-2}\}^{*}$ from equation (9) is exactly the number of $1$’s in the word $w_{k}[i+2:k+i]$. Since $w_{k}$ is constructed by the homomorphism $h$, it is easy to see that $|v_{i}|\geq(k-3)/2$. Note that no letter occurs more than once in $v_{i}$, hence $g(v_{i})=|v_{i}|$. Further, each substring of length $2\lceil\log k\rceil+2$ occurs at most once in $v_{1},\dots,v_{k}$, because otherwise there would be a factor of length $\lceil\log k\rceil$ occurring more than once in $B_{\lceil\log k\rceil}$. It follows that there are at least

[TABLE]

different factors of length $2\lceil\log k\rceil+2\in\Theta(\log k)$ in the right-hand side of the start rule of $\mathbb{G}$. By Lemma 2.1 (point 5) it follows that a smallest SLP for the right-hand side of the start rule has size $\Omega(k^{2}/\log k)$ and therefore $|\mathsf{RePair}(s_{k})|\in\Omega(k^{2}/\log k)$. (end proof of Claim 2)

In conclusion: We showed that a smallest SLP for $s_{k}$ has size $\mathcal{O}(k)$, while RePair produces an SLP of size $\Omega(k^{2}/\log k)$. This implies $\alpha_{\mathsf{RePair}}(s_{k})\in\Omega(k/\log k)$, which together with $n=|s_{k}|$ and $k\in\Theta(\log n)$ finishes the proof. ∎

Note that in the above proof, RePair chooses in the first $k-1$ rounds a digram for the replaced maximal string. Therefore, Theorem 3.7 also holds for the RePair-variant, where in every round a digram (which is not necessarily a maximal string) is replaced.

The goal of this section is to prove the following result:

Theorem 3.8**.**

Let $c\geq 1$ be a constant. If there exists a polynomial time grammar-based compressor $\mathcal{C}$ with $\alpha_{\mathcal{C}}(2,n)\leq c$ then there exists a polynomial time grammar-based compressor $\mathcal{D}$ with $\alpha_{\mathcal{D}}(n)\leq 6c$.

For a factor $24+\varepsilon$ (with $\varepsilon>0$) instead of 6 this result was shown in [2] using a more complicated block encoding.

We split the proof of Theorem 3.8 into two lemmas that state translations between SLPs over arbitrary alphabets and SLPs over a binary alphabet. For the rest of this section fix the alphabets $\Sigma=\{c_{0},\ldots,c_{k-1}\}$ and $\Sigma_{2}=\{a,b\}$. To translate between these two alphabets, we define an injective homomorphism $\varphi\colon\Sigma^{*}\to\Sigma_{2}^{*}$ by

[TABLE]

Lemma 3.9**.**

Let $w\in\Sigma^{*}$ be such that every symbol from $\Sigma$ occurs in $w$. From an SLP $\mathbb{A}$ for $w$ one can construct in polynomial time an SLP $\mathbb{B}$ for $\varphi(w)$ of size at most $3\cdot|\mathbb{A}|$.

Proof.

To translate $\mathbb{A}$ into an SLP $\mathbb{B}$ for $\varphi(w)$, we first add the productions $A_{0}\to b$ and $A_{i}\to aA_{i-1}$ for every $i,1\leq i\leq k-1$. Finally, we replace in $\mathbb{A}$ every occurrence of $c_{i}\in\Sigma$ by $A_{i}$. This yields an SLP $\mathbb{B}$ for $\varphi(w)$ of size $|\mathbb{A}|+2k-1$. Because $k\leq|\mathbb{A}|$ (since every symbol from $\Sigma$ occurs in $w$), we obtain $|\mathbb{B}|\leq 3\cdot|\mathbb{A}|$. ∎

Lemma 3.10**.**

Let $w\in\Sigma^{*}$ such that every symbol from $\Sigma$ occurs in $w$. From an SLP $\mathbb{B}$ for $\varphi(w)$ one can construct in polynomial time an SLP $\mathbb{A}$ for $w$ of size at most $2\cdot|\mathbb{B}|$.

Proof.

Let $\mathbb{B}=(N,\Sigma_{2},P,S)$ be an SLP for $\varphi(w)$, where $w\in\Sigma^{*}$. We can assume that every right-hand side of $\mathbb{B}$ is a non-empty string. Consider a nonterminal $A\in N$ of $\mathbb{B}$. Since $\mathbb{B}$ produces $\varphi(w)$, $A$ produces a factor of $\varphi(w)$, which is a word from $\{a,b\}^{*}$. We cannot directly translate $\mathrm{val}(A)$ back to a word over $\Sigma^{*}$ because $\mathrm{val}(A)$ does not have to belong to the image of $\varphi$. But $\mathrm{val}(A)$ is a factor of a string from $\varphi(\Sigma^{*})$. Note that a string over $\{a,b\}$ is a factor of a string from $\varphi(\Sigma^{*})$ if and only if it does not contain a factor $a^{i}$ with $i\geq k$. Let $\mathrm{val}(A)=a^{i_{1}}b\cdots a^{i_{n}}ba^{i_{n+1}}$ be such a string, where $n\geq 0$, and $0\leq i_{1},\ldots,i_{n+1}<k$. We factorize $\mathrm{val}(A)$ into three parts in the following way. If $n=0$ (i.e., $\mathrm{val}(A)=a^{i_{1}}$) then we split $\mathrm{val}(A)$ into $\varepsilon$, $\varepsilon$, and $a^{i_{1}}$. If $n>0$ then we split $\mathrm{val}(A)$ into $a^{i_{1}}b$, $a^{i_{2}}b\cdots a^{i_{n}}b$, and $a^{i_{n+1}}$. Let us explain the intuition behind this factorization. We concentrate on the case $n>0$; the case $n=0$ is simpler. Note that irrespective of the context in which an occurrence of $\mathrm{val}(A)$ appears in $\mathrm{val}(\mathbb{B})$, we can translate the middle part $a^{i_{2}}b\cdots a^{i_{n}}b$ into $c_{i_{2}}\cdots c_{i_{n}}$. We will therefore introduce in the SLP $\mathbb{A}$ for $w$ a variable $A^{\prime}$ that produces $c_{i_{2}}\cdots c_{i_{n}}$. For the left part $a^{i_{1}}b$ we can not directly produce $c_{i_{1}}$ because an occurrence of $\mathrm{val}(A)$ could be preceded by an $a$-block $a^{i_{0}}$, yielding the symbol $c_{i_{0}+i_{1}}$. Therefore, the algorithm that produces $\mathbb{A}$ will only memorize the symbol $c_{i_{1}}$ without writing it directly on a right-hand side of an $\mathbb{A}$-production. Similarly, the algorithm will memorize the length $i_{n+1}$ of the final $a$-block of $\mathrm{val}(A)$.

Let us now come to the formal details of the proof. As usual, we write $\mathbb{Z}_{k}$ for $\{0,1,\ldots,k-1\}$ and w.l.o.g. we assume that $\Sigma\cap\mathbb{Z}_{k}=\emptyset$. Consider a word $s=a^{i_{1}}b\cdots a^{i_{n}}ba^{i_{n+1}}$, where $n\geq 0$, and $0\leq i_{1},\ldots,i_{n+1}<k$. Motivated by the above discussion, we define $\ell(s)\in\Sigma\cup\{\varepsilon\}$, $m(s)\in\Sigma^{*}$ and $r(s)\in\mathbb{Z}_{k}$ as follows:

[TABLE]

Note that $\ell(s)=\varepsilon$ implies $m(s)=\varepsilon$. Finally, we define the word $\psi(s)\in\Sigma^{*}\mathbb{Z}_{k}$ as

[TABLE]

For a nonterminal $A\in N$ we define $\ell(A)=\ell(\mathrm{val}(A))$, $m(A)=m(\mathrm{val}(A))$ and $r(A)=r(\mathrm{val}(A))$. We now define an SLP $\mathbb{A}^{\prime}$ that contains for every nonterminal $A\in N$ a nonterminal $A^{\prime}$ such that $\mathrm{val}(A^{\prime})=m(A)$. Moreover, the algorithm also computes $\ell(A)\in\Sigma\cup\{\varepsilon\}$ and $r(A)\in\mathbb{Z}_{k}$.

We define the productions of $\mathbb{A}^{\prime}$ inductively over the structure of $\mathbb{B}$. Consider a production $(A\to\alpha)\in P$, where $\alpha=v_{0}A_{1}v_{1}A_{2}\cdots v_{n-1}A_{n}v_{n}\neq\varepsilon$ with $n\geq 0$, $A_{1},\ldots,A_{n}\in N$, and $v_{0},v_{1},\ldots,v_{n}\in\Sigma_{2}^{*}$. Let $\ell_{i}=\ell(A_{i})\in\Sigma\cup\{\varepsilon\}$ and $r_{i}=r(A_{i})\in\mathbb{Z}_{k}$, which have already been computed. The right-hand side for $A^{\prime}$ is obtained as follows. We start with the word

[TABLE]

Note that each of the factors $\ell_{i}A^{\prime}_{i}r_{i}$ produces (by induction) $\psi(\mathrm{val}(A_{i}))$. Next we remove every $A^{\prime}_{i}$ that derives the empty word (which is equivalent to $m(A_{i})=\varepsilon$). After this step, every occurrence of a symbol $i\in\mathbb{Z}_{k}$ in (11) is either the last symbol of the above word or it is followed by a symbol from $\mathbb{Z}_{k}\cup\Sigma$ (but not followed by a nonterminal $A^{\prime}_{j}$). To see this, recall that $\ell_{j}=\varepsilon$ implies $m(A_{j})=\varepsilon$, in which case $A^{\prime}_{j}$ is removed in (11).

The above fact allows us to eliminate all occurrences of symbols $i\in\mathbb{Z}_{k}$ in (11) except for the last one using the two reduction rules $i\,j\to i+j$ for $i,j\in\mathbb{Z}_{k}$ (which corresponds to $a^{i}a^{j}=a^{i+j}$) and $i\,c_{j}\to c_{i+j}$ (which corresponds to $a^{i}a^{j}b=a^{i+j}b$). If we perform these rules as long as possible (the order of applications is not relevant since these rules form a confluent and terminating system), only a single occurrence of a symbol $i\in\mathbb{Z}_{k}$ at the end of the string will remain. The resulting string $\alpha^{\prime}$ produces $\psi(A)$. Hence, we obtain the right-hand side for the nonterminal $A^{\prime}$ by removing the first symbol of $\alpha^{\prime}$ if it is from $\Sigma$ (this symbol is then $\ell(A)$) and the last symbol of $\alpha^{\prime}$, which must be from $\mathbb{Z}_{k}$ (this symbol is $r(A)$). Note that if $\alpha^{\prime}$ does not start with a symbol from $\Sigma$, then $\alpha^{\prime}$ belongs to $\mathbb{Z}_{k}$, in which case we have $\ell(A)=\varepsilon$.

Note that $\psi(\varphi(w))=w0$ for every $w\in\Sigma^{*}$, so for the start variable $S$ of $\mathbb{B}$ we must have $r(S)=0$, since $\mathrm{val}_{\mathbb{B}}(S)\in\varphi(\Sigma^{*})$. Let $S^{\prime}\to\sigma$ be the production for $S^{\prime}$ in $\mathbb{A}^{\prime}$. We obtain the SLP $\mathbb{A}$ by replacing this production by $S^{\prime}\to\ell(S)\sigma$. Since $\mathrm{val}_{\mathbb{A}^{\prime}}(S^{\prime})=m(S)$ and $\mathrm{val}_{\mathbb{B}}(S)=\varphi(w)$ we have $\mathrm{val}_{\mathbb{A}}(S^{\prime})=\ell(S)m(S)=w$.

To bound the size of $\mathbb{A}$ consider the word in (11) from which the right-hand side of the nonterminal $A^{\prime}$ is computed. All occurrences of symbols from $\mathbb{Z}_{k}$ are eliminated when forming this right-hand side. This leaves a word of length at most $|\alpha|+n$ (where $\alpha$ is the original right-hand side of the nonterminal $A$). The additive term $n$ comes from the symbols $\ell_{1},\ldots,\ell_{n}$. Hence, $|\mathbb{A}^{\prime}|$ is bounded by the size of $\mathbb{B}$ plus the total number of occurrences of nonterminals in right-hand sides of $\mathbb{B}$, which is at most $2|\mathbb{B}|-1$ (there is at least one terminal occurrence in a right-hand side). Since $|\mathbb{A}|=|\mathbb{A}^{\prime}|+1$ we get $|\mathbb{A}|\leq 2|\mathbb{B}|$.

The algorithm’s runtime for a production $A\to\alpha$ is linear in $|\alpha|$. This is because we start with the string (11) which can be computed in time $\mathcal{O}(|\alpha|)$. From this string, we remove all the $A_{i}^{\prime}$ that produce $\varepsilon$ and we also apply the two rewriting rules. Both of these can be done in a single left-to-right sweep over the string. The number of operations needed is linear in $|\alpha|$, where each operation needs constant time, i.e. removing an $A_{i}^{\prime}$ takes constant time, and using one of the rewriting rules also takes constant time. Since the algorithm uses the structure of $\mathbb{B}$ to visit each of its productions once, we overall obtain a linear running time in the size of $\mathbb{B}$. ∎

Example 3.11**.**

Consider the production $A\to a^{3}ba^{5}A_{1}a^{3}A_{2}a^{2}b^{2}A_{3}a^{2}$ and assume that $\mathrm{val}(A_{1})=a^{2}$, $\mathrm{val}(A_{2})=aba^{3}ba$ and $\mathrm{val}(A_{3})=ba^{2}ba^{3}$. Hence, when we produce the right-hand side for $A^{\prime}$ we have: $\mathrm{val}(A^{\prime}_{1})=\varepsilon$, $\mathrm{val}(A^{\prime}_{2})=c_{3}$, $\mathrm{val}(A^{\prime}_{3})=c_{2}$, $\ell_{1}=\varepsilon$, $r_{1}=2$, $\ell_{2}=c_{1}$, $r_{2}=1$, $\ell_{3}=c_{0}$, $r_{3}=3$. We start with the word (every digit is a single symbol)

[TABLE]

Then we replace $A^{\prime}_{1}$ by $\varepsilon$ and obtain $c_{3}\,5\,2\,3\,c_{1}A^{\prime}_{2}\,1\,c_{2}c_{0}\,0\,c_{0}A^{\prime}_{3}\,3\,2$. Applying the reduction rules finally yields $c_{3}c_{11}A^{\prime}_{2}c_{3}c_{0}c_{0}A^{\prime}_{3}5$. Hence, we have $\ell(A)=c_{3}$, $r(A)=5$ and the production for $A^{\prime}$ is $A^{\prime}\to c_{11}A^{\prime}_{2}c_{3}c_{0}c_{0}A^{\prime}_{3}$.

Proof of Theorem 3.8. Let $\mathcal{C}$ be an arbitrary grammar-based compressor working in polynomial time such that $\alpha_{\mathcal{C}}(2,n)\leq c$. The grammar-based compressor $\mathcal{D}$ works for an input word $w$ over an arbitrary alphabet as follows: Let $\Sigma=\{c_{0},\ldots,c_{k-1}\}$ be the set of symbols that occur in $w$ and let $\varphi$ be defined as in (10). Using $\mathcal{C}$, one first computes an SLP $\mathbb{B}$ for $\varphi(w)$ such that $|\mathbb{B}|\leq c\cdot g(\varphi(w))$. Then, using Lemma 3.10, one computes from $\mathbb{B}$ an SLP $\mathbb{A}$ for $w$ such that $|\mathbb{A}|\leq 2c\cdot g(\varphi(w))$. Lemma 3.9 implies $g(\varphi(w))\leq 3\cdot g(w)$ and hence $|\mathbb{A}|\leq 6c\cdot g(w)$, which proves the theorem. ∎

4. Hardness of grammar-based compression for binary alphabets

5. Open problems

One should try to narrow the gaps between the lower and upper bounds for the other grammar-based compressors analyzed in [7]. In particular, the gap between the known lower and upper bounds for the so-called global algorithms from [7] (like RePair) is still quite big. Charikar et al. [7] prove an upper bound $\Theta((n/\log n)^{2/3})$ for every global algorithm and nothing better is known for the three global algorithms RePair, LongestMatch, Greedy studied in [7]. Comparing to this upper bound, the known lower bounds are quite small: $\Omega(\log n/\log\log n)$ for RePair (by our Theorem 3.7), $\Omega(\log\log n)$ for longest match [7], and $1.348\ldots$. The latter is a very recent result from [17].111The table on page 2556 in [7] states the better lower bound of $1.37\ldots$, but the authors only show the lower bound $1.137\ldots$, see [7, Theorem 11].

Another open research problem is improving the constant 6 in Theorem 3.8. Recall that lowering this constant to at most $8569/8568$ would imply that the smallest grammar problem for binary strings cannot be solved in polynomial time unless $\mathsf{P}=\mathsf{NP}$.