Two-Dimensional Source Coding by Means of Subblock Enumeration

TL;DR

This paper extends the substring enumeration compression technique to two-dimensional sources like images by introducing a block-based approach using a flat torus model, reducing complexity and analyzing code length limits.

Contribution

It proposes a new 2D source coding method using block-by-block encoding with a flat torus model, improving efficiency over line-by-line methods.

Findings

Reduces encoding complexity for 2D sources

Uses flat torus as a probabilistic model

Analyzes average codeword length limits

Abstract

A technique of lossless compression via substring enumeration (CSE) attains compression ratios as well as popular lossless compressors for one-dimensional (1D) sources. The CSE utilizes a probabilistic model built from the circular string of an input source for encoding the source.The CSE is applicable to two-dimensional (2D) sources such as images by dealing with a line of pixels of 2D source as a symbol of an extended alphabet. At the initial step of the CSE encoding process, we need to output the number of occurrences of all symbols of the extended alphabet, so that the time complexity increase exponentially when the size of source becomes large. To reduce the time complexity, we propose a new CSE which can encode a 2D source in block-by-block instead of line-by-line. The proposed CSE utilizes the flat torus of an input 2D source as a probabilistic model for encoding the source instead of the circular string of the source. Moreover, we analyze the limit of the average codeword length of the proposed CSE for general sources.

Equations

$${\bm{p}}_{(i,j)}^{(i\!+\!k\!-\!1,j\!+\!l\!-\!1)}\!:=\!\left\{\begin{array}[]{ll}\lambda^{[0,l]}\ \,(k\!\leq\!0\text{ and }l\!\geq\!0),&\\ \lambda^{[k,0]}\ (k\!\geq\!0\text{ and }l\!\leq\!0),&\\ \left(\begin{array}[]{ccc}p_{(i,j)}&\cdots&p_{(i,j\!+\!l\!-\!1)}\\ \vdots&\ddots&\vdots\\ p_{(i\!+\!k\!-\!1,j)}&\cdots&p_{(i\!+\!k\!-\!1,j\!+\!l\!-\!1)}\end{array}\right)&\\ \ \ \ \ \ \ \ (k\!>\!0\text{ and }l\!>\!0)&\\ \end{array}\right.$$

$$\mathcal{D}(\mathbf{p}):=\{{\mathbf{p}}_{(i,j)}^{(i+k-1,j+l-1)}\text{ s.t. }$$

$$0\le k\le m-i+1,0\le l\le n-j+1\}.$$

$$[\mathbf{p}]:=\{\mathbf{q}\in {\mathcal{X}}^{[m,n]}\text{ s.t. }\mathbf{q}\in \mathcal{D}(\bar{\mathbf{p}})\}.$$

$$N(\mathbf{u}|\mathbf{p})$$

$$\sum _{\mathbf{u}\in {\mathcal{X}}^{[k,l]}}N(\mathbf{u})=mn.$$

$$N(\mathbf{v})$$

$$N({\mathbf{v}}^{\prime })$$

$$\mathcal{T}(\mathbf{p},k,l)$$

$${}^{\forall }\mathbf{w}\in {\mathcal{X}}^{[k,l]},\mathbf{q}\text{ is primitive.}\}$$

$$\mathcal{B}(\mathbf{p}):=\{\mathbf{b}\in {\mathcal{X}}^{[k,l]}\text{ s.t. }$$

$$1\le k\le m,1\le l\le n\}\cup \{{\lambda }^{[0,0]}\}.$$

$$\mathcal{T}(\mathcal{B}(\mathbf{p}),\mathbf{p},i):=\{$$

$$1\le {}^{\forall }j\le i,\mathbf{q}\text{ is primitive.}\}$$

$$(E(n),e({\mathbf{b}}_2,{\mathbf{b}}_3,\dots ,{\mathbf{b}}_{|\mathcal{B}(\mathbf{x})|}),ϵ(\text{rank(x)})).$$

$$\min (N(\mathbf{a}:\mathbf{w}),$$

$$N(\mathbf{w})-N(\mathbf{w}:\mathbf{c}))\ge 1.$$

$$0\le N({\mathbf{b}}_i)\le n-1.$$

$$\max \{0,N(\mathbf{a}:\mathbf{w})-\sum _{\mathbf{d}\in \hat{\mathcal{X}}\backslash \{\mathbf{c}\}}N(\mathbf{w}:\mathbf{d}),N(\mathbf{w}:\mathbf{c})-\sum _{\mathbf{b}\in \hat{\mathcal{X}}\backslash \{\mathbf{a}\}}N(\mathbf{b}:\mathbf{w})\}$$

$$\le N(\mathbf{a}:\mathbf{w}:\mathbf{c})\le \min \{N(\mathbf{a}:\mathbf{w}),N(\mathbf{w}:\mathbf{c})\}.$$

$$\displaystyle\frac{1}{n}\ \$$

$$\displaystyle\frac{1}{I({\bm{b}}_{i})}\ \$$

$$\displaystyle\frac{|\mathcal{T}(\mathcal{B}({\bm{x}}),{\bm{x}},i)|}{|\mathcal{T}(\mathcal{B}({\bm{x}}),{\bm{x}},i\!-\!1)|}\ \$$

$${\mathcal{B}}_0(\mathbf{p})$$

$${\mathcal{B}}_1(\mathbf{p})$$

$${\mathcal{B}}_2(\mathbf{p})$$

$${\mathcal{B}}_3(\mathbf{p})$$

$$\min (N(\mathbf{e}/\mathbf{v}),$$

$$N(\mathbf{v})-N(\mathbf{v}/\mathbf{g}))\ge 1.$$

$$\max \{0,N(\mathbf{e}/\mathbf{v})-\sum _{\mathbf{h}\in {\mathcal{X}}^{[1,|\mathbf{e}{|}_c]}\backslash \{\mathbf{g}\}}N(\mathbf{v}/\mathbf{h}),N(\mathbf{v}/\mathbf{g})-\sum _{\mathbf{f}{\in }^{[1,|\mathbf{e}{|}_c]}\backslash \{\mathbf{e}\}}N(\mathbf{f}/\mathbf{v})\}$$

$$\le N(\mathbf{e}/\mathbf{v}/\mathbf{g})\le \min \{N(\mathbf{e}/\mathbf{v}),N(\mathbf{v}/\mathbf{g})\}.$$

$$\displaystyle\frac{1}{mn}\ \$$

$$\displaystyle\max\left(\frac{1}{I({\bm{b}}_{i})},\frac{1}{I^{\prime}({\bm{b}}_{i})}\right)\ \$$

$$\displaystyle\frac{|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)|}{|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i\!-\!1)|}\ \$$

$$(E(m),E(n),e({\mathbf{b}}_2,{\mathbf{b}}_3,\dots ,{\mathbf{b}}_{|\mathcal{B}(\mathbf{p})|}),ϵ(\text{rank(p)})).$$

$$\mathbf{X}:=\{X^{[m,n]}=(X_{(1,1)}^{<m,n>},X_{(1,2)}^{<m,n>},\dots ,X_{(m,n)}^{<m,n>}){\}}_{m=1,n=1}^{\infty ,\infty }$$

$$\hat{H}(\mathbf{X}):=\underset{m\to \infty ,n\to \infty }{\mathrm{limsup}}\frac{1}{mn}H(X^{[m,n]}).$$

$$\underset{m,n\to \infty }{\mathrm{limsup}}E\left[\frac{\ell (X^{[m,n]})}{mn}\right]=\hat{H}(\mathbf{X}).$$

Full text

I Introduction

In 2010, Dubé and Beaudoin proposed an efficient off-line data compression algorithm for a binary source known as Compression via Substring Enumeration (CSE) [1]. In [2], Yokoo proposed a universal CSE algorithm for a binary source and various versions of the CSE for a binary source have been proposed so far [3, 4, 5]. It is reported that performance of the CSE [4] is as well as that of an efficient off-line data compression algorithm using the Burrows-Wheeler transformation (BWT) [6]. In [7], it is proved that an encoder, which is a deterministic finite automaton, of the CSE and an encoder without sinks of the antidictionary coding [8] are isomorphic for a binary source. Moreover, an antidictionary coding proposed in [9] provided the first CSE for $q$-ary ($q\!>\!2$) alphabet sources as a byproduct. Iwata and Arimura proposed the modified algorithm and evaluated the maximum redundancy rate of the CSE for the $k$th order Markov sources [10].

For encoding an input source, the CSE utilizes a probabilistic model built from the circular string which is obtained by concatenating the first symbol to the last symbol of the source. A probabilistic model of the circular string is also useful for the BWT and antidictionary coding [7, 9], and in [11], it is shown that an antidictionary built from the circular string is useful for genome comparison such as deoxyribonucleic acid (DNA). However, for a 2D source such as an image, computational time of the CSE is exponential with respect to line length since the CSE works in line-by-line. The CSE deals with a line of 2D source as a symbol of an extended alphabet. At the initial step of the CSE encoding process, the CSE needs to output frequencies of all symbols of the extended alphabet.

To reduce the computational time, we propose a new CSE for a 2D source which utilizes the flat torus of an input 2D source as a probabilistic model instead of the circular string of the source. In the initial step, the total number of output blocks is constant since the new CSE works in block-by-block. Moreover, we evaluate the limit of the average codeword length of the proposed algorithm for general sources.

II Basic Notations and Definitions

II-A Alphabet and Block

Let ${\mathcal{X}}$ be a finite source alphabet $\{0,1,\dots,J\!-\!1\}$ and let $|{\mathcal{X}}|$ be a cardinality of ${\mathcal{X}}$, that is $|{\mathcal{X}}|=J$. Let ${\mathcal{X}}^{[m,n]}$ be the set of all $m\!\times\!n$ finite blocks ${\bm{p}}=(p_{(i,j)})_{1\leq i\leq m,1\leq j\leq n}$ over ${\mathcal{X}}$, where $p_{(i,j)}\in{\mathcal{X}}$ is the element of ${\bm{p}}$ at $(i,j)$-coordinate. Furthermore, let ${\mathcal{X}}^{[*,*]}$ be $\cup_{m,n\geq 0}{\mathcal{X}}^{[m,n]},$ where ${\mathcal{X}}^{[m,n]}$ includes the empty block $\lambda^{[m,n]}$ when at least one of $m$ and $n$ is [math]. For convenience, ${\mathcal{X}}^{[m,0]}$ and ${\mathcal{X}}^{[0,n]}$ are defined as $\{\lambda^{[m,0]}\}$ and $\{\lambda^{[0,n]}\}$, respectively. For ${\bm{p}}\in{\mathcal{X}}^{[*,*]}$, let $|{\bm{p}}|_{r}$ and $|{\bm{p}}|_{c}$ be the length of row (the height) and the length of column (the width), respectively. For example, when ${\mathcal{X}}=\{0,1\}$, Fig. 2 illustrates ${\bm{p}}\in{\mathcal{X}}^{[3,3]}$ where $|{\bm{p}}|_{r}\!=\!|{\bm{p}}|_{c}\!=\!3$.

II-B Subblock, Concatenation, and Dictionary

For ${\bm{p}}\!\in\!{\mathcal{X}}^{[m,n]}$, a subblock ${\bm{p}}_{(i,j)}^{(i\!+\!k\!-\!1,j\!+\!l\!-\!1)}\!\!\!\in\!\!{\mathcal{X}}^{[k,l]}$ is defined as

[TABLE]

where $1\!\leq\!i\!\leq\!m$, $1\!\leq\!j\!\leq\!n$, $k\!\leq\!m\!-\!i\!+\!1$, and $l\!\leq\!n\!-\!j\!+\!1$. Hereinafter, without notice, we assume that the height and width of ${\bm{p}}$ are respectively given by $m~{}(\geq 2)$ and $n~{}(\geq 2)$. In particular, $(m-1)\times n$ subblocks ${\bm{p}}_{(1,1)}^{(m\!-\!1,n)}$ and ${\bm{p}}_{(2,1)}^{(m,n)}$ are denoted by $\pi_{r}({\bm{p}})$ and $\sigma_{r}({\bm{p}})$, respectively. Moreover, $m\times(n-1)$ subblocks ${\bm{p}}_{(1,1)}^{(m,n\!-\!1)}$ and ${\bm{p}}_{(1,2)}^{(m,n)}$ are denoted by $\pi_{c}({\bm{p}})$ and $\sigma_{c}({\bm{p}})$, respectively. For example, for ${\bm{p}}$ in Fig. 2, Fig. 2 shows $\pi_{c}({\bm{p}})$, $\sigma_{c}({\bm{p}})$, $\pi_{r}({\bm{p}})$, and $\sigma_{r}({\bm{p}})$ from the left-hand side.

For ${\bm{p}}$, the dictionary of ${\bm{p}}$ is defined as the set of all the subblocks of ${\bm{p}}$, that is,

[TABLE]

Now we define a concatenation of blocks by column-wisely as follows: For two blocks ${\bm{s}},{\bm{t}}\!\in\!{\mathcal{X}}^{[*,*]}$ such that $|{\bm{s}}|_{r}\!=\!|{\bm{t}}|_{r}$, define ${\bm{s}}\!:\!{\bm{t}}\in{\mathcal{X}}^{[|{\bm{s}}|_{r},|{\bm{s}}|_{c}+|{\bm{t}}|_{c}]}$ to be a block obtained by concatenating ${\bm{t}}$ at the end of ${\bm{s}}$ in columns. Similarly, we define a concatenation of blocks by row-wisely as follows: for two blocks ${\bm{u}},{\bm{v}}\!\in\!{\mathcal{X}}^{[*,*]}$ such that $|{\bm{u}}|_{c}\!=\!|{\bm{v}}|_{c}$, define ${\bm{u}}\!/\!{\bm{v}}\in{\mathcal{X}}^{[|{\bm{u}}|_{r}+|{\bm{v}}|_{r},|{\bm{u}}|_{c}]}$ to be a block obtained by concatenating ${\bm{u}}$ at the end of ${\bm{v}}$ in rows.

II-C Flat Torus, Primitive, and Frequencies of Subblocks

For ${\bm{p}}$, a flat torus of ${\bm{p}}$, denoted by ${\bm{p}}^{T}$, is constructed by concatenating the most left-hand side column (resp. the top row) to the most right-hand side column (resp. the bottom row) of ${\bm{p}}$. The flat torus can be treated as an infinite pattern such that $p_{(i,j)}=p^{T}_{(i+km,j+ln)}$ for non-negative integer $k,l$.

For ${\bm{q}}\in{\mathcal{X}}^{[m,n]}$ and $\bar{{\bm{p}}}:=({\bm{p}}\!:\!{\bm{p}})/({\bm{p}}\!:\!{\bm{p}})$, if there exist positive integers $i~{}(1\leq i\leq m)$ and $j~{}(1\leq j\leq n)$ such that ${\bm{q}}=\bar{{\bm{p}}}_{(i,j)}^{(i+m-1,j+n-1)}$ is satisfied, then the equivalence relation is denoted as ${\bm{q}}\simeq{\bm{p}}$. Note that $\bar{{\bm{p}}}$ is a $2m\times 2n$ subblock of ${\bm{p}}^{T}$. Let $[{\bm{p}}]$ be the set of all the blocks ${\bm{q}}$ such that ${\bm{q}}\simeq{\bm{p}}$,

[TABLE]

If $|\,[{\bm{p}}]\,|=mn$, ${\bm{p}}$ is called primitive. Hereinafter, without notice, we assume that ${\bm{p}}$ is primitive. For example, ${\bm{p}}$ shown in Fig. 2 is primitive.

For ${\bm{p}}$ and ${\bm{u}}\in{\mathcal{X}}^{[k,l]}$ ($0\!\leq\!k\!\leq\!m$ and $0\!\leq\!l\!\leq\!n)$,

[TABLE]

where $N(\lambda^{[k,l]}|{\bm{p}})=mn$ ($k=0$ or $l=0$). For convenience, we often adopt the notation $N({\bm{u}})$ instead of $N({\bm{u}}|{\bm{p}})$. For ${\bm{p}}$, $0\!\leq\!k\!\leq\!m$, and $0\!\leq\!l\!\leq\!n$,

[TABLE]

Moreover, for ${\bm{v}}\in{\mathcal{X}}^{[i,j]}~{}(0\!\leq\!i\!\leq\!m,0\!\leq\!j\!<\!n)$ and ${\bm{v}}^{\prime}\in{\mathcal{X}}^{[k,l]}~{}(0\!\leq\!k\!<\!m,0\!\leq\!l\leq\!n)$,

[TABLE]

II-D Classifications of Flat Tori and Core

For ${\bm{p}}$ and $k~{}(0\leq k\leq m)$, and $l~{}(0\leq l\leq n)$,

[TABLE]

For example, $[{\bm{p}}]=\mathcal{T}({\bm{p}},m,n)$. For $0\!\leq\!k\!<\!n$ and fixed $0\!\leq\!l\!\leq\!n$, $\mathcal{T}({\bm{p}},k,l)$ is monotone decreasing with $k$, that is $\mathcal{T}({\bm{p}},k\!+\!1,l)\!\subset\!\mathcal{T}({\bm{p}},k,l)$. Similarly, for fixed $0\!\leq\!k^{\prime}\!\leq\!n$ and $0\!\leq\!l^{\prime}\!<\!n$, $\mathcal{T}({\bm{p}},k^{\prime},l^{\prime}\!+\!1)\!\subset\!\mathcal{T}({\bm{p}},k^{\prime},l^{\prime})$. Next, we define $\mathcal{B}({\bm{p}})$,

[TABLE]

We assume that elements of $\mathcal{B}({\bm{p}})$ are ordered in ascending order with its height (if heights of the elements are equal, then the elements ordered with its width; if widths of the elements are equal, then the elements are ordered in lexicographical order column-wisely) where ${\bm{b}}_{i}$ is the $i$th element of $\mathcal{B}({\bm{p}})~{}(1\!\leq\!i\!\leq\!|\mathcal{B}({\bm{p}})|)$. For $i~{}(1\!\leq\!i\!\leq\!|\mathcal{B}({\bm{p}})|)$,

[TABLE]

For example, $[{\bm{p}}]\!=\!\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},|\mathcal{B}({\bm{p}})|)$. For $1\!\leq\!i\!<\!|\mathcal{B}({\bm{p}})|$, $\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)$ is monotone decreasing with $i$, that is $\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i\!+\!1)\!\subset\!\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)$.

A ${\bm{u}}\!\in\!\mathcal{B}({\bm{p}})$ such that ${\bm{a}}:{\bm{u}},{\bm{b}}:{\bm{u}},{\bm{u}}:{\bm{c}},{\bm{u}}:{\bm{d}}\!\in\!\mathcal{D}(\bar{{\bm{p}}})$ where ${\bm{a}},{\bm{b}}(\neq\!{\bm{a}}),{\bm{c}},{\bm{d}}(\neq\!{\bm{c}})\!\in\!{\mathcal{X}}^{[|{\bm{u}}|_{r},1]}$ is called c-core. A ${\bm{v}}\in\mathcal{B}({\bm{p}})$ such that ${\bm{e}}/{\bm{v}},{\bm{f}}/{\bm{v}},{\bm{v}}/{\bm{g}},{\bm{v}}/{\bm{h}}\in\mathcal{D}(\bar{{\bm{p}}})$ where ${\bm{e}},{\bm{f}}(\neq\!{\bm{e}}),{\bm{g}},{\bm{h}}(\neq\!{\bm{g}})\!\in\!{\mathcal{X}}^{[1,|{\bm{v}}|_{c}]}$ is called r-core.

III Review of Conventional CSE

The conventional CSE is a lossless compression algorithm for a 1D source. For ${\bm{p}}$, we can regard ${\bm{p}}$ as a 1D source ${\bm{x}}\in\hat{{\mathcal{X}}}^{[1,n]}$ over an extended alphabet $\hat{{\mathcal{X}}}(={\mathcal{X}}^{[m,1]})$, so that the CSE can encode ${\bm{p}}$ as a 1D source ${\bm{x}}$. For ${\bm{x}}$, the CSE outputs a following triplet

[TABLE]

In (9), $E(n)$ represents an encoded $n$ by means of Elias integer code [12]. And rank(${\bm{x}}$) represents an index for identifying ${\bm{x}}$ in $[{\bm{x}}]$ such as the rank of ${\bm{x}}$ in $[{\bm{x}}]$ with lexicographical order. Then, $\epsilon$(rank(${\bm{x}}$)) represents an encoded rank(${\bm{x}}$) by $\lceil\log_{2}n\rceil$ bits, and $e({\bm{b}}_{2},{\bm{b}}_{3},\dots,{\bm{b}}_{|\mathcal{B}({\bm{x}})|})$ represents a sequence of $N({\bm{b}}_{i})~{}(2\leq i\leq|\mathcal{B}({\bm{x}})|)$ which are encoded by an entropy coding where $N({\bm{b}}_{i})$ represents $N({\bm{b}}_{i}|{\bm{x}})$ in this subsection. In encoding, for ${\bm{b}}_{i}\in\mathcal{B}({\bm{x}})$, $i$ is selected from 2 to $|\mathcal{B}({\bm{x}})|$ since $N({\bm{b}}_{1})=N(\lambda^{[0,0]})=n$ and $n$ is encoded as $E(n)$. For $2\leq i\leq|\mathcal{B}({\bm{x}})|$,

(C-i)

in case of $|{\bm{b}}_{i}|_{c}\!=\!1$: Encode $N({\bm{b}}_{i})$ if ${\bm{b}}_{i}\neq{\bm{b}}_{|\hat{{\mathcal{X}}}|\!+\!1}$,

(C-ii)

in case of $|{\bm{b}}_{i}|_{c}\!\geq\!2$: Encode $N({\bm{b}}_{i})$ if (10) holds and ${\bm{a}},{\bm{c}}\!\in\!\hat{{\mathcal{X}}}\!\backslash\{{\bm{b}}_{|\hat{{\mathcal{X}}}|\!+\!1}\}$ where ${\bm{b}}_{i}={\bm{a}}\!:\!{\bm{w}}\!:\!{\bm{c}}$ such that ${\bm{w}}=\sigma_{c}(\pi_{c}({\bm{b}}_{i}))$

where ${\bm{b}}_{|\hat{{\mathcal{X}}}|\!+\!1}$ is the element of $\hat{{\mathcal{X}}}$ having the largest index in $\mathcal{B}({\bm{x}})$ and note that (10) was first shown in [10]. Note that in (C-i), $N({\bm{b}}_{i})$ is encoded even if $N({\bm{b}}_{i})=0$.

In (C-i), $N({\bm{b}}_{|\hat{{\mathcal{X}}}|\!+\!1})$ can be calculated by using (3) and already encoded ${\bm{b}}_{j}(j<|\hat{{\mathcal{X}}}|\!+\!1)$. Similarly, in (C-ii), $N({\bm{b}}_{i})$ such that ${\bm{a}}={\bm{b}}_{|\hat{{\mathcal{X}}}|\!+\!1}$ or ${\bm{c}}={\bm{b}}_{|\hat{{\mathcal{X}}}|\!+\!1}$ can be calculated by using (4) and ${\bm{b}}_{k}~{}(k<i)$. Therefore, they are not encoded.

[TABLE]

As for ${\bm{b}}_{i}(={\bm{a}}\!:\!{\bm{w}}\!:\!{\bm{c}})$ in (C-ii), satisfying (10) is the same that ${\bm{w}}$ is a c-core. Moreover, since ${\bm{a}},{\bm{w}},{\bm{c}}\!\in\!\mathcal{D}(\bar{{\bm{x}}})$ and (3) holds, number of candidates of ${\bm{b}}_{i}$ for encoding in (C-ii) is polynomial order with $n$. The details are described in the bottom of this section. In (C-i), $N({\bm{b}}_{i})$ satisfies the following inequality

[TABLE]

In (C-ii), $N({\bm{b}}_{i})$ satisfies the following inequality [9]

[TABLE]

The left-hand side term in (10) is given by the difference between the 3rd term and the 1st term in (12). Therefore, if (10) does not hold, then the 1st and the 3rd terms are equal. In other words, $N({\bm{b}}_{i})=\min\{N({\bm{a}}\!:\!{\bm{w}}),N({\bm{w}}\!:\!{\bm{c}})\}$ holds, so that $N({\bm{b}}_{i})$ can be calculated. Hence, $N({\bm{b}}_{i})$ is not encoded if (10) does not hold.

Let $I({\bm{a}}\!:\!{\bm{w}}\!:\!{\bm{c}})$ be $\min(N({\bm{a}}\!:\!{\bm{w}}),N({\bm{w}}\!:\!{\bm{c}}),N({\bm{w}})\!-\!N({\bm{a}}\!:\!{\bm{w}}),$ $N({\bm{w}})\!-\!N({\bm{w}}\!:\!{\bm{c}}))+1$ where $\min(\cdot)$ is the left-hand term of (10). For encoding $N({\bm{b}}_{i})$ by an entropy coding, a probability is assigned to $N({\bm{b}}_{i})$ as follows [2].

[TABLE]

The assigned probabilities are encoded by an entropy coding such as an arithmetic coding [13].

For encoding 2D source ${\bm{p}}$ by the conventional CSE, there is a problem with respect to computational time. In (C-i), number of encoded $N({\bm{b}}_{i})~{}(2\leq i\leq|\hat{{\mathcal{X}}}|)$ is exponential with respect to $m$ since $|\hat{{\mathcal{X}}}|$ is $|{\mathcal{X}}|^{m}$. In practical, $m$ is greater than 1000 for an image ${\bm{p}}\in{\mathcal{X}}^{[m,n]}$, so that the number is greater than $2^{1000}$ even if $|{\mathcal{X}}|=2$. Note that in (C-ii), number of encoded $N({\bm{b}}_{i})$ is not exponential with respect to $m$ and $n$. The reason is as follows. Since ${\bm{w}}$ is a c-core, from (3) and (4), the total number of c-cores is polynomial order with respect to $m$ and $n$. Moreover, since $N({\bm{a}}{\bm{w}})\geq 1$ and $N({\bm{w}}{\bm{c}})\geq 1$ in (10), ${\bm{a}},{\bm{c}}\in{\mathcal{D}}(\bar{{\bm{x}}})\cap\hat{{\mathcal{X}}}$ also hold. From (3) and (4), $|{\mathcal{D}}(\bar{{\bm{x}}})\cap\hat{{\mathcal{X}}}|$ never exceeds $mn$. Hence, the total number of candidates ${\bm{b}}_{i}(={\bm{a}}\!:\!{\bm{w}}\!:\!{\bm{c}})$ for encoding in (C-ii) is polynomial order with respect to $m$ and $n$. In other words, the set of all the candidates can be utilized instead of $\mathcal{B}({\bm{x}})$ in (C-ii) in practice. Note that $\mathcal{B}({\bm{x}})$ is utilized for simplifying the explanation in this paper. As for compression ratio, only a relation on column is utilized as shown in (10) and a relation on row is not utilized.

IV Proposed Algorithm

For ${\bm{p}}$, we assume that $m\leq n$. Let $K$ and $L$ be $\lfloor\sqrt{\log_{|{\mathcal{X}}|}\log_{|{\mathcal{X}}|}m}\rfloor$ and $\lfloor\sqrt{\log_{|{\mathcal{X}}|}\log_{|{\mathcal{X}}|}n}\rfloor$, respectively.

We divide $\mathcal{B}({\bm{p}})$ into four disjoint parts with respect to size of its elements.

[TABLE]

Elements of $\mathcal{B}_{i}({\bm{p}})~{}(i=0,1,2,3)$ are ordered in ascending order with its height (if heights of the elements are equal, then the elements ordered with its width; if widths of the elements are equal, then the elements are ordered in lexicographical column-wisely.) Then, elements of $\mathcal{B}({\bm{p}})$ are reordered with $(\mathcal{B}_{0}({\bm{p}}),\mathcal{B}_{1}({\bm{p}}),\mathcal{B}_{2}({\bm{p}}),\mathcal{B}_{3}({\bm{p}}))$. For $2\leq i\leq|\mathcal{B}({\bm{p}})|$,

(P-i)

in case of ${\bm{b}}_{i}\!\in\!\mathcal{B}_{1}({\bm{p}})$: Encode $N({\bm{b}}_{i})$ if ${\bm{b}}_{i}\neq J\!-\!1$,

(P-ii)

in case of ${\bm{b}}_{i}\!\in\!\mathcal{B}_{2}({\bm{p}})\!\cup\!\mathcal{B}_{3}({\bm{p}})$:

**     1)**

if $|{\bm{b}}_{i}|_{c}\!=\!1$: Encode $N({\bm{b}}_{i})$ if (10) holds and ${\bm{a}},{\bm{c}}\!\in\!{\mathcal{X}}\backslash\{J\!-\!1\}$ where ${\bm{b}}_{i}\!=\!{\bm{a}}\!\!:\!\!{\bm{w}}\!\!:\!\!{\bm{c}}$ such that ${\bm{w}}\!=\!\sigma_{c}(\pi_{c}({\bm{b}}_{i}))$,

**     2)**

if $|{\bm{b}}_{i}|_{r}\!=\!1$: Encode $N({\bm{b}}_{i})$ if (16) holds and ${\bm{e}},{\bm{g}}\!\in\!{\mathcal{X}}\backslash\{J\!-\!1\}$ where ${\bm{b}}_{i}\!=\!{\bm{e}}\!/\!{\bm{v}}\!/\!{\bm{g}}$ such that ${\bm{v}}\!=\!\sigma_{r}(\pi_{r}({\bm{b}}_{i}))$,

**     3)**

if $|{\bm{b}}_{i}|_{c}\geq 2$ and $|{\bm{b}}_{i}|_{r}\geq 2$: Encode $N({\bm{b}}_{i})$ if both (10) and (16) hold where ${\bm{a}},{\bm{c}}\!\in\!{\mathcal{X}}^{[|{\bm{b}}_{i}|_{r},1]}\backslash\{{\bm{x}}(|{\bm{b}}_{i}|_{r},1)\}$ and ${\bm{e}},{\bm{g}}\!\in\!{\mathcal{X}}^{[1,|{\bm{b}}_{i}|_{c}]}\backslash\{{\bm{x}}(1,|{\bm{b}}_{i}|_{c})\}$,

where ${\bm{x}}(k,1)$ and ${\bm{x}}(1,l)$ are the element of ${\mathcal{X}}^{[k,1]}$ and ${\mathcal{X}}^{[1,l]}$ having the largest index in $\mathcal{B}({\bm{p}})$, respectively.

[TABLE]

As for ${\bm{b}}_{i}(={\bm{e}}\!/\!{\bm{v}}\!/\!{\bm{g}})$ in 2) and 3), satisfying (16) is the same that ${\bm{v}}$ is a r-core. As shown in the discussions in Sec. III, number of candidates of ${\bm{b}}_{i}$ for encoding in (P-ii) is polynomial order with $m$ and $n$. The details are described in the bottom of this section.

The conventional CSE utilizes only condition (10) with respect to column, while the proposed algorithm utilizes conditions (10) and (16) with respect to column and row, respectively, for encoding ${\bm{p}}$. In 1) and 2), ${\bm{b}}_{i}$ is one row and one column, so that (10) and (16) is only utilized, respectively. In (P-i), $N({\bm{b}}_{i})$ satisfies $0\!\leq\!N({\bm{b}}_{i})\leq\!mn\!-\!1$. In (P-ii), $N({\bm{b}}_{i})$ such that $|{\bm{b}}_{i}|_{c}\geq 2$ satisfies a modified (12) which is obtained by replacing $\hat{{\mathcal{X}}}$ by ${\mathcal{X}}^{[|{\bm{a}}|_{r},1]}$, and $N({\bm{b}}_{i})$ such that $|{\bm{b}}_{i}|_{r}\geq 2$ satisfies the following inequality

[TABLE]

As described on (10), similarly, the left-hand side term in (16) is given by the difference between the 3rd term and the 1st term in (17). Therefore, if (16) does not hold, then the 1st and the 3rd terms are equal. In other words, $N({\bm{b}}_{i})=\min\{N({\bm{e}}\!/\!{\bm{v}}),N({\bm{v}}\!/\!{\bm{g}})\}$ holds, so that $N({\bm{b}}_{i})$ can be calculated. Hence, $N({\bm{b}}_{i})$ is not encoded if (16) does not hold. Therefore, in 3), $N({\bm{b}}_{i})$ is encoded if both (10) and (16) hold.

Let $I^{\prime}({\bm{e}}\!/\!{\bm{v}}\!/\!{\bm{g}})$ be $\min(N({\bm{e}}\!/\!{\bm{v}}),N({\bm{v}}\!/\!{\bm{g}}),N({\bm{v}})\!-\!N({\bm{e}}\!/\!{\bm{v}}),$ $N({\bm{v}})\!-\!N({\bm{v}}\!/\!{\bm{g}}))+1$ where $\min(\cdot)$ is the left-hand term of (16). For encoding $N({\bm{b}}_{i})$ by an entropy coding, a probability is assigned to $N({\bm{b}}_{i})$ as follows.

[TABLE]

The assigned probabilities are encoded by an entropy coding such as an arithmetic coding. For ${\bm{p}}$, the proposed algorithm outputs a following quartet

[TABLE]

In (21), $E(m)$ and $E(n)$ represent encoded $m$ and $n$ by means of Elias integer code, respectively. And rank(${\bm{p}}$) represents an index for identifying ${\bm{p}}$ in $[{\bm{p}}]$ such as the rank of ${\bm{p}}$ in $[{\bm{p}}]$ with lexicographical order column-wisely. Then, $\epsilon$(rank(${\bm{p}}$)) represents an encoded rank(${\bm{p}}$) by $\lceil\log_{2}mn\rceil$ bits, and $e({\bm{b}}_{2},{\bm{b}}_{3},\dots,{\bm{b}}_{|\mathcal{B}({\bm{p}})|})$ represents a sequence of $N({\bm{b}}_{i})~{}(2\leq i\leq|\mathcal{B}({\bm{p}})|)$ which are encoded by an entropy coding as described in Sec III.

In the proposed algorithm, in (P-i), number of encoded $N({\bm{b}}_{i})$ is $|{\mathcal{X}}|\!-\!1$, that is a constant, while that in (C-i) is exponential with respect to $m$, that is $|{\mathcal{X}}|^{m}\!-\!1$. As for (P-ii), number of candidates $N({\bm{b}}_{i})$ for encoding is polynomial order with respect to $m$ and $n$. The reason is as follows. As for 1), it is the same as (C-ii). As for 2) and 3), since ${\bm{v}}$ is a r-core, from the discussions on a c-core described in Sec. III, the total number of candidates $N({\bm{b}}_{i})$ for encoding is polynomial order with $m$ and $n$. In other words, the set of all the candidates can be utilized instead of $\mathcal{B}({\bm{p}})$ in (P-ii) in practice. Similarly, note that $\mathcal{B}({\bm{p}})$ is utilized for simplifying the explanation in this paper. Hence, for a 2D source ${\bm{p}}$, the total number of output blocks of the proposed algorithm is polynomial with respect to $m$ and $n$ while that of the conventional CSE is exponential with respect to $m$.

V Evaluation of the Proposed Algorithm

A general source $\mathbf{X}$ is defined as

[TABLE]

where a random variable $X^{[m,n]}$ takes a value in the $m\times n$ Cartesian product ${\mathcal{X}}^{[m,n]}$ of ${\mathcal{X}}$ [14]. The probability distribution of a random variable $X^{[m,n]}$ is denoted by $P_{X^{[m,n]}}$. For $\mathbf{X}$, the sup-entropy rate of $\mathbf{X}$ is defined as

[TABLE]

For ${\bm{p}}$, let $\ell({\bm{p}})$ be a codeword length of the proposed algorithm. Let $\ell_{0}({\bm{p}})$ be the total codeword length of $E(m)$, $E(n)$, and $\epsilon$(rank(${\bm{p}}$)) in (21). The codeword length of $e({\bm{b}}_{2},{\bm{b}}_{3},\dots,{\bm{b}}_{|\mathcal{B}({\bm{p}})|})$ consists of three parts $\ell_{1}({\bm{p}})$, $\ell_{2}({\bm{p}})$, and $\ell_{3}({\bm{p}})$ where $\ell_{1}({\bm{p}})$, $\ell_{2}({\bm{p}})$, and $\ell_{3}({\bm{p}})$ are the total codeword length of ${N}({\bm{b}}_{i})$ for ${\bm{b}}_{i}\in\mathcal{B}_{1}({\bm{p}})$, ${\bm{b}}_{i}\in\mathcal{B}_{2}({\bm{p}})$, and ${\bm{b}}_{i}\in\mathcal{B}_{3}({\bm{p}})$, respectively. Here, $\ell({\bm{p}})=\ell_{0}({\bm{p}})+\ell_{1}({\bm{p}})+\ell_{2}({\bm{p}})+\ell_{3}({\bm{p}})$.

Theorem 1 is one of our main results. To prove Theorem 1, we show three lemmas. Lemma 2 is a 2D version of Lemma 3 [2], and the proofs of Lemmas 2 and 3 are omitted in this paper.

Theorem 1

For a general source $\mathbf{X}$,

[TABLE]

Lemma 2

For ${\bm{p}}$, $1\!\leq\!k\!\leq\!m$, and $1\!\leq\!l\!\leq\!n$

[TABLE]

Lemma 3

If ${\bm{b}}_{i+1}\in\mathcal{B}({\bm{p}})$ such that $|{\bm{b}}_{i+1}|_{c}\geq 2$ does not satisfy (10) or such that $|{\bm{b}}_{i+1}|_{r}\geq 2$ does not satisfy (16), then $\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i+1)=\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i).$

Lemma 4

[TABLE]

Proof.

For ${\bm{w}}\in{\mathcal{X}}^{[K,L]}$, $P_{X^{[m,n]}}({\bm{w}})$ can be written by

[TABLE]

where $m^{\prime}$ and $n^{\prime}$ are $m\!-\!K\!+\!1$ and $n\!-\!L\!+\!1$, respectively, and $(i,j)$ is a coordinate. For ${\bm{p}}$, let $N^{\prime}({\bm{w}}\,|\,{\bm{p}})$ be $|\{(i,j)\text{ s.t. }{\bm{p}}_{(i,j)}^{(i\!+\!K\!-\!1,j\!+\!L\!-\!1)}\!=\!{\bm{w}},1\!\leq\!i\!\leq\!m^{\prime},1\!\leq\!j\!\leq\!n^{\prime}\}|$. Moreover, $\frac{N({\bm{w}}\,|\,{\bm{p}})}{mn}$ can be written by $\left(\frac{N^{\prime}({\bm{w}}\,|\,{\bm{p}})+\delta}{m^{\prime}n^{\prime}}\right)\left(\frac{m^{\prime}n^{\prime}}{mn}\right)$ where $0\!\leq\!\delta\!\leq\!(K\!-\!1)(n\!-\!L\!+\!1)\!+\!(L\!-\!1)m$ from (2). Since $K$ and $L$ are respectively $\lfloor\sqrt{\log_{|{\mathcal{X}}|}\log_{|{\mathcal{X}}|}m}\rfloor$ and $\lfloor\sqrt{\log_{|{\mathcal{X}}|}\log_{|{\mathcal{X}}|}n}\rfloor$, $\frac{N({\bm{w}}|{\bm{p}})}{mn}$ converges to $\frac{N^{\prime}({\bm{w}}|{\bm{p}})}{m^{\prime}n^{\prime}}$ as $m$ and $n$ go to infinity. Since $E\left[\frac{N^{\prime}({\bm{w}}|X^{[m,n]})}{m^{\prime}n^{\prime}}\right]=P_{X^{[m,n]}}({\bm{w}})$,

[TABLE]

∎

(Proof of Theorem 1).

As for $\ell_{0}({\bm{p}})$, from the assumption, since $m\leq n$, $\ell_{0}({\bm{p}})\!\leq 2(\log_{2}n\!+\!2\log_{2}\log_{2}n\!+\!7)\!+\!\lceil\log_{2}mn\rceil$ where $(\log_{2}n\!+\!2\log_{2}\log_{2}n\!+\!7)$ and $\lceil\log_{2}mn\rceil$ are costs of Elias integer code for $n$ and $\epsilon$(rank(${\bm{p}}$)), respectively. As for $\ell_{1}({\bm{p}})$, the cost of $N({\bm{b}}_{i})$ in (P-i) is $\lceil\log_{2}mn\rceil$ bits from (18), so that $\ell_{1}({\bm{p}})\!\leq\!(|{\mathcal{X}}|\!-\!1)\lceil\log_{2}mn\rceil$. As for $\ell_{2}({\bm{p}})$, since $I({\bm{b}}_{i})\leq mn$ and $I^{\prime}({\bm{b}}_{i})\leq mn$, costs of $I({\bm{b}}_{i})$ and $I^{\prime}({\bm{b}}_{i})$ are at most $\log_{2}mn$ bits. Moreover, since $m\leq n$ and $K\leq L$,

[TABLE]

Therefore,

[TABLE]

As for $\ell_{3}({\bm{p}})$, from (20), cost of $N({\bm{b}}_{i})$ is $-\log_{2}(|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)|/|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i\!-\!1)|)$ bits.

Cost of the next encoded $N({\bm{b}}_{j})$ such that $N({\bm{b}}_{i})$ has been encoded immediately before $N({\bm{b}}_{j})$ is $-\log_{2}(|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},j)|/|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},j\!-\!1)|)$. From Lemma 3, $|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},j\!-\!1)|\!=\!|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)|$. Therefore, $N({\bm{b}}_{j})$ can be written by $-\log_{2}(|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},j)|/|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)|)$, Hence, the denominator $|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)|$ for ${\bm{p}}_{j}$ is equal to the previous numerator $|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},i)|$ for ${\bm{b}}_{i}$, so that they are canceled. Moreover, since $|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},|\mathcal{B}({\bm{p}})|)|\!=\!|[{\bm{p}}]|\!=\!\!mn$,

[TABLE]

where $S$ is the index of the first block ${\bm{b}}_{S}\in\mathcal{B}_{3}({\bm{p}})$ which is encoded by arithmetic coding. From Lemma 3, $|\mathcal{T}(\mathcal{B}({\bm{p}}),{\bm{p}},S\!-\!1)|\!=\!|\mathcal{T}({\bm{p}},K,L)|$. Therefore,

[TABLE]

From (25) and Lemma 2,

[TABLE]

Therefore,

[TABLE]

From Jensen’s inequality, $E[\frac{N({\bm{w}}|X^{[m,n]})}{mn}]E[\log_{2}\frac{N({\bm{w}}|X^{[m,n]})}{mn}]\leq E[\frac{N({\bm{w}}|X^{[m,n]})}{mn}\log_{2}\frac{N({\bm{w}}|X^{[m,n]})}{mn}]$. Therefore, from Lemma 4,

[TABLE]

From (23) and (27),

[TABLE]

The proposed code is a prefix code, so that Kraft’s inequality is satisfied. Therefore, $\limsup_{m,n\rightarrow\infty}E\left[\frac{\ell(X^{[m,n]})}{mn}\right]\!\geq\!\hat{H}(\mathbf{X})$. ∎

From Remark 1.7.3 [14], if $\mathbf{X}$ is a stationary source, $\hat{H}(\mathbf{X})$ can be expressed by $H(\mathbf{X})(:=\lim_{m,n\rightarrow\infty}\frac{H(X^{[m,n]})}{mn})$, that is the entropy rate of $\mathbf{X}$. Therefore, if $\mathbf{X}$ is a stationary source, the average codeword length of the proposed algorithm converges to $H(\mathbf{X})$ as $m$ and $n$ go to infinity.

VI Conclusion

For reducing computational time, we proposed a new CSE for a 2D source which utilizes the flat torus of the source while the conventional CSE utilizes the circular string of the source as a probabilistic model. The total number of output blocks of the new CSE is polynomial while that of the conventional CSE is exponential with respect to the source size. The new CSE encodes the source in block-by-block while the conventional CSE does in line-by-line. Moreover, we prove that an upper bound on the average codeword length of the proposed CSE converges to the sup-entropy rate for a general source as size of the input source goes to infinity. Furthermore, if a general source is a stationary source, then the length converges to the entropy rate of the source as the size goes to infinity.