Optimality in Quantum Data Compression using Dynamical Entropy

TL;DR

This paper explores lossless quantum data compression for general quantum state sequences, establishing that optimal compression length equals the quantum dynamical entropy of an associated quantum Markov chain.

Contribution

It introduces quantum stochastic ensembles and links the optimal compression length to quantum dynamical entropy, extending previous i.i.d. assumptions.

Findings

Optimal average codeword length equals quantum dynamical entropy.

Quantum stochastic ensembles generalize i.i.d. models.

Provides a theoretical foundation for quantum data compression.

Abstract

In this article we study lossless compression of strings of pure quantum states of indeterminate-length quantum codes which were introduced by Schumacher and Westmoreland. Past work has assumed that the strings of quantum data are prepared to be encoded in an independent and identically distributed way. We introduce the notion of quantum stochastic ensembles, allowing us to consider strings of quantum states prepared in a more general way. For any identically distributed quantum stochastic ensemble we define an associated quantum Markov chain and prove that the optimal average codeword length via lossless coding is equal to the quantum dynamical entropy of the associated quantum Markov chain.

Equations

$$H(X)=-\sum _{x\in S}p(x){\log }_{2}p(x).$$

$$S(\rho )=-\text{tr}(\rho {\log }_2\rho ).$$

$$EL(C):=\sum _{x\in S}p(x)\ell (C(x))=\mathbb{E}[\ell (C(X))],$$

$$C^{+}(x_1{x}_2\cdots x_n)=C(x_1)C(x_2)\cdots C(x_n)\text{for all }{x}_1{x}_2\cdots x_n\in S^n\text{ and }n\in \mathbb{N},$$

$$\sum _{i=1}^m{2}^{-{\ell }_i}\le 1.$$

$$\begin{array}{rl}{C}_{\text{opt}}:& ={\mathrm{argmin}}_C\{EL(C):C\text{ is uniquely decodable}\}\\ & ={\mathrm{argmin}}_C\{EL(C):\text{the codeword lengths of the }C\text{ satisfy}\sum _i{2}^{-{\ell }_i}\le 1\},\end{array}$$

$$H(X)\le E{L}^{\ast }(X)<H(X)+1,$$

$$H(X_1,X_2,\dots ,X_n)\le E{L}^{\ast }(X_1,X_2,\dots ,X_n)<H(X_1,X_2,\dots ,X_n)+1.$$

$$E{L}_n^{\ast }(\mathbf{X}):=\frac{1}{n}E{L}^{\ast }(X_1,X_2,\dots ,X_n)$$

$$H(\mathbf{X})=\underset{n\to \infty }{\lim }\frac{1}{n}H(X_1,\dots ,X_n),$$

$$\frac{H(X_1,X_2,\dots ,X_n)}{n}\le E{L}_n^{\ast }(\mathbf{X})<\frac{H(X_1,X_2,\dots ,X_n)}{n}+\frac{1}{n}.$$

$$E{L}_n^{\ast }(\mathbf{X})\to H(\mathbf{X})\text{as }n\to \infty .$$

$$E{L}_n^{\ast }(\mathbf{X})\to H(X)\text{as }n\to \infty .$$

$$U^{+}(|s_1{s}_2\cdots s_n⟩)=U(|s_1⟩)U(|s_2⟩)\cdots U(|s_n⟩),$$

$$U=\sum _{i=1}^d|C(x_i)⟩⟨e_i|.$$

$$U^{\ell }=\sum _{i_1=1}^d\cdots \sum _{i_{\ell }=1}^d|C(x_{i_1})C(x_{i_2})\cdots C(x_{i_{\ell }})⟩⟨e_{i_1}{e}_{i_2}\cdots e_{i_{\ell }}|$$

$$U^{+}=\sum _{\ell =0}^{\infty }{U}^{\ell },$$

$$\Lambda :=\sum _{\ell =0}^{\infty }\ell {\Pi }_{\ell },$$

$$U=\sum _{i=1}^d|{\psi }_i⟩⟨e_i|.$$

$$\mathrm{tr}(U^{†}{2}^{-\Lambda }U)\le 1.$$

$$\ell (|\omega ⟩):=⟨\omega |\Lambda |\omega ⟩=⟨Us,\Lambda Us⟩=⟨s,U^{†}\Lambda Us⟩.$$

$$\ell (|\omega ⟩)=\sum _{i=1}^d|⟨e_i|s⟩{|}^2{\ell }_i,$$

$${\rho }_{\mathcal{S}}=\sum _{n=1}^N{p}_n|s_n⟩⟨s_n|\in S_1(H_{\mathcal{S}}).$$

$$EL(U)=\mathrm{tr}({\rho }_{\mathcal{S}}{U}^{†}\Lambda U).$$

$$\begin{array}{rl}{U}_{\text{opt}}:& ={\mathrm{argmin}}_U\{EL(U):U\text{ is uniquely decodable with length eigenstates}\}\\ & ={\mathrm{argmin}}_U\{EL(U):\mathrm{tr}(U^{†}{2}^{-\Lambda }U)\le 1\}\\ & ={\mathrm{argmin}}_U\{EL(U):U\text{ is a c-q scheme satisfying }\sum _{i=1}^d{2}^{-{\ell }_i}\le 1\},\end{array}$$

$$E{L}^{\ast }({\rho }_{\mathcal{S}}):=EL(U_{\text{opt}})=\mathrm{tr}({\rho }_{\mathcal{S}}{U}_{\text{opt}}^{†}\Lambda U_{\text{opt}}).$$

$$S({\rho }_{\mathcal{S}})\le E{L}^{\ast }({\rho }_{\mathcal{S}})<S({\rho }_{\mathcal{S}})+1.$$

$$\sum _{n_{k+1}=1}^{N}p(n_1,\cdots ,n_k,n_{k+1})=p(n_1,\cdots ,n_k),$$

$$E{L}_k^{\ast }({\rho }_{{\mathcal{S}}^k})=\frac{1}{k}E{L}^{\ast }({\rho }_{{\mathcal{S}}^k})$$

$$\frac{1}{k}S({\rho }_{{\mathcal{S}}^k})\le E{L}_k^{\ast }({\rho }_{{\mathcal{S}}^k})<\frac{1}{k}S({\rho }_{{\mathcal{S}}^k})+\frac{1}{k}.$$

Full text

This work is part of the second author’s PhD thesis.

The second author was partially supported by a SPARC Graduate Research Grant from the Office of the Vice President for Research at the University of South Carolina.

1. Introduction

In the theory of data compression of classical information theory one wishes to encode a symbol set, $S$, with a code, $C$, which is a mapping from the symbol set $S$ to the set $A^{+}$ of all finite strings (or sequences) of elements from the alphabet $A$, where $A$ is usually taken to be the binary alphabet $\{0,1\}$. The set $A^{+}$ is frequently referred to as the codebook and its elements are called codewords. Since we compress long strings (sequences) of messages, concatenation is used to extend the code $C$ to the set $S^{+}$ containing all finite strings from the symbol set $S$. This extension of $C$ is denoted by $C^{+}$ and it is called the extended code. A code $C$ is said to be uniquely decodable if its extended code is an injective function. In that case, the decoding function is the inverse of $C^{+}$. If each symbol $x$ of the symbol set $S$ that we wish to encode is always prepared with the same probability $p(x)$, independent of the string of symbols that have appeared earlier, then the sequence $(X_{n})_{n\in\mathbb{N}}$ of random variables which gives us the string of symbols to be encoded is independent and identically distributed (i.i.d.) with values in the symbol set $S$ with probability mass function equal to $(p(x))_{x\in S}$. If $X$ denotes any member of this sequence of random variables then its Shannon entropy $H(X)$ is defined as

[TABLE]

In Shannon’s original works on the subject ([19, 20]), the Noiseless Coding Theorem was proved which states that, for any $\delta>0$, $(H(X)+\delta)$-many binary bits per symbol are sufficient in order to encode strings of symbols if each entry of the sequence is prepared in a i.i.d. way, with probability of error tending to zero as the length of the strings tend to infinity. Moreover, Shannon showed that for any $R<H(X)$, if at most $R$ bits are used per symbol, then the probability of error tends to 1 as the length of the strings tend to infinity. Thus the Shannon entropy $H(X)$ can be interpreted as the minimum expected number of binary bits per symbol that are necessary in order to encode strings of symbols with arbitrarily small error (i.e.  asymptotically lossless coding) given that the elements of the string of symbols are encoded in an i.i.d. way.

The setting of quantum data compression for indeterminate-length quantum codes is similar to the setting of classical data compression. In this case, the symbol set $\mathcal{S}$ contains the symbol states which are normalized vectors spanning a Hilbert space $H_{\mathcal{S}}$. Here we only consider the compression of pure quantum states, therefore we restrict our attention to normalized vectors or pure states. The classical binary alphabet $A=\{0,1\}$ is replaced by the set of qubits $\mathcal{A}=\{|0\rangle,|1\rangle\}$ which is the standard orthonormal basis of the Hilbert space $\mathcal{H}_{\mathcal{A}}=\mathbb{C}^{2}$. The classical codebook $A^{+}$ is replaced by the free Fock space $H_{\mathcal{A}}^{\oplus}=\oplus_{\ell=0}^{\infty}H_{\mathcal{A}}^{\otimes\ell}$. A quantum code is a linear isometry $U:H_{\mathcal{S}}\to H_{\mathcal{A}}^{\oplus}$, and the corresponding extended code is a map $U^{+}$ which is defined on the free Fock space $H_{\mathcal{S}}^{\oplus}=\oplus_{\ell=0}^{\infty}H_{\mathcal{S}}^{\otimes\ell}$ by “concatenation;” i.e. tensor products of the values of $U$ in the free Fock space $H_{\mathcal{A}}^{\oplus}$. The quantum code $U$ is called uniquely decodable if $U^{+}$ is also an isometry.

The Noiseless Coding Theorem was extended to indeterminate-length quantum codes in 1995 by Schumacher [17]. Schumacher showed that, for any $\delta>0$, $(S(\rho)+\delta)$-many qubits per symbol are sufficient in order to encode strings of symbol states if each entry of the sequence is prepared in a i.i.d. way, with probability of error tending to zero as the length of the strings tends to infinity. Here $\rho=\rho_{\mathcal{S}}$ is the ensemble state representing the quantum ensemble $\mathcal{S}$, and $S(\rho)$ is the von Neumann entropy of the density matrix $\rho$ given by

[TABLE]

Moreover, Schumacher showed that for any $R<S(\rho)$, if at most $R$ qubits are used per symbol, then the probability of error tends to 1 as the length of the strings tends to infinity. Thus the von Neumann entropy $S(\rho)$ can be interpreted as the minimum expected number of qubits per symbol that are necessary in order to encode strings of symbol states with arbitrarily small error (i.e. asymptotically lossless coding) given that the elements of the string of symbol states are prepared in an i.i.d. way.

Indeterminate-length quantum codes were considered by Schumacher and Westmoreland in [18], and later by Müller, Rogers and Nagarajan in [13, 14]; and Bellomo, Bosyk, Holik and Zozor in [5]. In all three of these papers, the authors prove a version of the quantum Kraft-McMillan Theorem which states that every uniquely decodable quantum code must satisfy an inequality in terms of the lengths of its eigenstates. Their presentations are very similar to that of the classical Kraft-McMillan Theorem ([8, Theorems 5.2.1 and 5.5.1]) except that these authors did not provide a converse statement. In Theorem 2.9, we present a modified version of the quantum Kraft-McMillan Theorem giving a converse statement, thus characterizing the uniquely decodable quantum codes. Our Theorem 2.9 comes in handy when we define an optimal quantum code that corresponds to a given ensemble.

In Subsection 2.2 we introduce the notion of quantum stochastic ensemble and Markov ensemble, allowing us to prepare strings of symbol states for quantum data compression such that the appearance of each symbol in the string may depend on the previous symbols; i.e. the strings of symbol states are not necessarily prepared in an i.i.d. way. Quantum sources that emit sequences of quantum symbols that are not necessarily statistically independent have been considered in the literature [9] and they are well suited for quantum communications. A stochastic ensemble is a sequence $(\mathcal{S}^{k})_{k\in\mathbb{N}}$, where $\mathcal{S}^{k}=\{p(n_{1},\ldots,n_{k}),|s_{n_{1}}\cdots s_{n_{k}}\rangle\}_{n_{1},\ldots,n_{k}=1}^{N}$ for each $k\in\mathbb{N}$ such that $p$ is the probability mass function of a discrete stochastic process $\mathbf{X}$, $\{|s_{n}\rangle\}_{n=1}^{N}$ is a collection of vector states referred to as the symbol states and $p(n_{1},\ldots,n_{k})$ is the probability that the string of quantum symbols $|s_{n_{1}}\cdots s_{n_{k}}\rangle$ is encoded, for each $k\in\mathbb{N}$ and $n_{1},\ldots,n_{k}\in\{1,\ldots,N\}$.

Our main results, Theorems 3.4 and 3.8, give quantum dynamical entropy interpretations for the average minimum codeword length per symbol as the length of strings of symbol states tend to infinity when the coding is assumed to be lossless. These results extend the result of Schumacher [17] and Bellomo et al. [5] which state that for an i.i.d. prepared quantum ensemble the optimal codeword length per symbol is equal to the von Neumann entropy of the initial ensemble state for asymptotically lossless coding. In our result we use the quantum Markov chain (QMC) approach to quantum dynamical entropy which we recall in Subsection 3.2. The notion of QMC was introduced by Accardi in [1] and its use for describing dynamical entropy was first appeared in [3] in terms of the Accardi-Ohya-Watanabe (AOW) entropy. Another QMC approach was introduced by Tuyls in [21] for the study of the Alicki-Fannes (AF) entropy, which was introduced in [4] and often referred to as ALF entropy to emphasize Lindblad’s contributions. Finally, a generalization of both QMC approaches was given in [10], where the authors introduced the Kossakowski-Ohya-Watanabe (KOW) entropy. Throughout this article, we will follow mainly the terminology and notations of [3] and [10].

2. Data Compression

In what follows, all codings will be done into strings of bits or strings of qubits for classical and quantum codes, respectively. Therefore all codewords will be strings of elements from a binary alphabet $A=\{0,1\}$ (in the classical case) or, possibly the superposition of, strings from a quantum binary alphabet $\mathcal{A}=\{|0\rangle,|1\rangle\}$ which is an orthonormal basis of the Hilbert space $H_{\mathcal{A}}=\mathbb{C}^{2}$ (in the quantum case). The extensions to $d$-bits or $d$-qubits can easily be done in both cases.

2.1. Classical Codes and the Kraft Inequality

Let $S$ be a finite or countable set equipped with the power set $\sigma$-algebra $\mathcal{P}(S)$, and let $X$ be a random variable with values in $S$. The set $S$ will be referred to as the symbol set that we wish to encode. In the literature, the set $S$ is referred to as the set of objects, the message set, or sometimes even the index set. For any set $Y$, we will set $Y^{+}$ equal to the set $\cup_{\ell=0}^{\infty}Y^{\ell}$ which is the collection of all possible finite strings from $Y$, where $Y^{0}$ denotes the empty set (or empty string). Lastly, let $A=\{0,1\}$ be the binary alphabet. A code $C:S\rightarrow A^{+}$ is a mapping from $S$ to $A^{+}$, the set of finite strings with letters in the binary alphabet $A$. The range of the code, $A^{+}$, is referred to as the codebook and its elements are the codewords. Moreover, for each $x\in S$, we refer to $C(x)$ as the codeword of the symbol $x$. For each $a\in A^{+}$, we call the length of $a$ (denoted by $\ell(a)$) the unique integer $m$ such that $a\in A^{m}$.

The expected length of a code $C$ on a symbol set $S$ is given by

[TABLE]

where $p:S\rightarrow[0,1]$ is the probability mass function (pmf) of the random variable $X$ and the expectation $\mathbb{E}$ is taken with respect to $p$.

We extend the code $C$ by concatenation to obtain the extended code, also called the extension of $C$, $C^{+}:S^{+}\rightarrow A^{+}$. That is to say

[TABLE]

and we define $C^{+}(\emptyset)=\emptyset$. We call the code $C$ uniquely decodable whenever its extension $C^{+}$ is injective; i.e. $C$ is uniquely decodable whenever all strings of symbols from $S$ are pairwise distinguishable. In lossless coding we are only interested in uniquely decodable codes.

An extremely useful class of uniquely decodable codes are the so-called instantaneous (or prefix-free) codes. A code is said to be prefix-free if no codeword is the prefix of another; i.e. for every distinct pair $x,y\in S$ there is no $a\in A^{+}$ such that $C(x)a=C(y)$. Prefix-free codes are called instantaneous because the decoder is able to read out each codeword from a string of codewords, instantaneously, as soon as she sees that word appear in a string (without waiting for the entire string).

The Kraft-McMillan Inequality is fundamental in classical data compression.

Theorem 2.1**.**

(Kraft-McMillan Inequality, [8, Theorems 5.2.1 and 5.5.1]) For any uniquely decodable code over a symbol set $S$ with cardinality $|S|=m\in\mathbb{N}$, the codeword lengths $\ell_{1},\ell_{2},\ldots,\ell_{m}$ must satisfy the inequality

[TABLE]

Conversely, given a set of codeword lengths that satisfies this inequality, there exists an instantaneous code with these code lengths.

Remark 2.2**.**

The Kraft-McMillan Inequality is sometimes referred to only as the Kraft Inequality. This is due to the fact that Kraft was the first to prove the inequality in [11], although his original result refers only to instantaneous codes. McMillan later extended Kraft’s work to include all uniquely decodable codes in [12]. Furthermore, it is worth noting that the Kraft-McMillan inequality can be extended to a countable set of symbols (see Theorem 5.2.2 and the corollary following Theorem 5.5.1 in [8]). When including countable sets of symbols, the inequality is referred to as the Extended Kraft-McMillan Inequality.

An immediate corollary to the Kraft-McMillan Inequality is the following:

Corollary 2.3**.**

Given any uniquely decodable code with codeword lengths $\ell_{1},\ell_{2},\ldots,\ell_{m}$, there exists an instantaneous code with these same code lengths.

We call a uniquely decodable code $C$ optimal whenever the expected length $EL(C)$ is minimized; i.e. the optimal uniquely decodable code is given by

[TABLE]

where the last equality follows from Theorem 2.1. We set $EL^{*}(X):=EL(C_{\text{opt}})$ the optimal expected length of the random variable $X$. The results for the optimal expected length are summarized in the following:

Theorem 2.4**.**

([8, Theorem 5.4.1]) Let $X$ be a random variable with range in the symbol set $S$. Then the optimal expected length of $X$ satisfies the inequality

[TABLE]

where $H(X)$ is the Shannon entropy of $X$, i.e. $H(X)=-\sum_{i\in S}p_{i}\log_{2}p_{i}$ where $(p_{i})_{i\in S}$ is the pmf of $X$.

Well known examples of codes which satisfy the inequality of Theorem 2.4 are the so-called Huffman codes and Shannon-Fano codes.

In the above theorem, we are only interested in the compressability of single codewords. Suppose instead that we wish to compress strings of codewords with code distributions given by a stochastic process $\mathbf{X}=(X_{i})_{i=1}^{\infty}$. Then, for each $n\in\mathbb{N}$, Theorem 2.4 holds for the random vector $(X_{1},X_{2},\ldots,X_{n})$, giving

[TABLE]

For each $n\in\mathbb{N}$, we set

[TABLE]

to be the optimal expected codeword length per symbol for the first $n$ symbols. We can then express the optimal expected codeword length per symbol (over all symbols) in terms of the entropy rate, which is a dynamical entropy for stochastic processes. The entropy rate of a stochastic process $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ is given by

[TABLE]

whenever the limit exists. There are many instances when it is known that the above limit exists (e.g. stationary stochastic processes, see [8, Theorem 4.2.1]).

Theorem 2.5**.**

([8, Theorem 5.4.2]) The optimal expected codeword length per symbol for a stochastic process $\mathbf{X}=(X_{i})_{i=1}^{\infty}$ satisfies

[TABLE]

Moreover, if $\mathbf{X}$ is such that the limit defining entropy rate exists (e.g. $\mathbf{X}$ is a stationary stochastic process), then

[TABLE]

In particular, if $\mathbf{X}$ consists of independent identically distributed (i.i.d.) copies of a random variable $X$, then

[TABLE]

This finishes our brief overview of data compression in classical information theory. For a more detailed exposition see [8, Chapter 5].

2.2. Quantum Data Compression

We begin with the description of indeterminate-length quantum codes, whose preliminary investigation began with Schumacher [16] and Braunstein et. al in [7], and they were formalized in [18]. We may think of the codes introduced in the previous section as being varying-length codes; the term indeterminate-length is used to draw attention to the fact that a quantum code must allow for superpositions of codewords, including those superpositions containing codewords with different lengths. We will follow mainly the formalisms in [5] as opposed to the zero-extended forms of [18]. A description of the connection between these two formalisms can be found in [6].

For any Hilbert space $H$, we will denote by $H^{\oplus}:=\oplus_{\ell=0}^{\infty}H^{\otimes\ell}$ the free Fock space of $H$, where $H^{\otimes 0}=\mathbb{C}$. We will denote the scalar $1\in H^{\otimes 0}$ by $|\emptyset\rangle$ and refer to it as the empty string. Let $\mathcal{S}=\{p_{n},|s_{n}\rangle\}_{n=1}^{N}$ be an ensemble of pure states, or simply ensemble, where $p=\{p_{n}\}_{n=1}^{N}$ is the pmf of a random variable $X$ and $|s_{n}\rangle$ is an element of a $d$-dimensional Hilbert space $H_{\mathcal{S}}$, for each $1\leq n\leq N$, such that $H_{\mathcal{S}}=\operatorname{span}\{|s_{n}\rangle\}_{n=1}^{N}$. The collection $\{|s_{n}\rangle\}_{n=1}^{N}$ will be referred to as the symbol states of the ensemble $\mathcal{S}$. An (indeterminate-length) quantum code, $U$, over a quantum binary alphabet $\mathcal{A}:=\{|0\rangle,|1\rangle\}$, which is an orthonormal basis for $H_{\mathcal{A}}=\mathbb{C}^{2}$, is a linear isometry $U:H_{\mathcal{S}}\rightarrow H_{\mathcal{A}}^{\oplus}$. The extended quantum code of $U$ is the linear mapping $U^{+}:H_{\mathcal{S}}^{\oplus}\rightarrow H_{\mathcal{A}}^{\oplus}$ given by

[TABLE]

for all $|s_{1}s_{2}\cdots s_{n}\rangle\in H_{\mathcal{S}}^{\otimes n}$ and $n\in\mathbb{N}$, and we set $U^{+}(|\emptyset\rangle)=|\emptyset\rangle$, where concatenation is defined according to [13, Definition 2.3] (see also [14, Section V]).

The quantum code $U$ is said to be uniquely decodable if the extended quantum code $U^{+}$ is an isometry. Throughout this paper, we will restrict ourselves only to the situation where the range of $U$ is a subset of $H_{\mathcal{A}}^{\oplus\ell_{\text{max}}}$ for some $\ell_{\text{max}}\in\mathbb{N}$; i.e. there is a finite upper bound $\ell_{\text{max}}$ on the length of all codewords.

Remark 2.6**.**

The authors of [6] allow non-empty strings to map to the empty string. In their paper, the authors send along a classical side channel to give the lengths of the codewords and so that convention is possible. Without the classical side channel (as is the approach in the present paper) allowing non-empty strings to map to the empty string will cause the quantum code to not be uniquely decodable.

Let $\mathcal{S}=\{p_{n},|s_{n}\rangle\}_{n=1}^{N}$ be an ensemble whose symbol states $\{|s_{n}\rangle\}_{n=1}^{N}$ span a Hilbert space $H_{\mathcal{S}}$ of dimension $d$. Consider a classical uniquely decodable code, $C$, on a symbol set, $S=\{x_{i}\}_{i=1}^{d}$, with $d$-many symbols. We will construct a corresponding uniquely decodable quantum code, $U$, from $C$ by identifying the classical binary alphabet $A=\{0,1\}$ with the quantum binary alphabet $\mathcal{A}=\{|0\rangle,|1\rangle\}\subseteq\mathbb{C}^{2}$ and the symbol set, $S$, with any orthonormal basis $\{|e_{i}\rangle\}_{i=1}^{d}$ of $H_{\mathcal{S}}$; this construction is given in [5]. Fix an orthonormal basis $\{|e_{i}\rangle\}_{i=1}^{d}$ of $H_{\mathcal{S}}$ and define the quantum code $U:H_{\mathcal{S}}\rightarrow H_{\mathcal{A}}^{\oplus}$ by the equation

[TABLE]

It is clear that $|C(x_{i})\rangle\in H_{\mathcal{A}}^{\otimes\ell_{i}}\subseteq H_{\mathcal{A}}^{\oplus}$, where $\ell_{i}$ is the length of $C(x_{i})$, and that $\{|C(x_{i})\rangle\}_{i=1}^{d}$ is an orthonormal set, so that $U$ is a linear isometry. Furthermore, since $C$ is uniquely decodable, the map $U^{\ell}:H_{\mathcal{S}}^{\otimes\ell}\rightarrow H_{\mathcal{A}}^{\oplus}$ defined by the equation

[TABLE]

is a linear isometry for each $\ell\in\mathbb{N}$. Since the extended quantum code $U^{+}:H_{\mathcal{S}}^{\oplus}\rightarrow H_{\mathcal{A}}^{\oplus}$ is given by

[TABLE]

we see that $U^{+}$ is a linear isometry and hence $U$ is uniquely decodable. We will refer to quantum codes constructed from classical ones by Equation (3) as classical-quantum encoding schemes (c-q schemes).

Remark 2.7**.**

Notice that the symbol states $\{|s_{n}\rangle\}_{n=1}^{N}$ of the ensemble $\mathcal{S}$ are not directly encoded by the $|C(x_{i})\rangle$’s unless $N=d$ and there exists a permutation $\sigma$ of $\{1,\ldots,d\}$ such that $|s_{\sigma(i)}\rangle=|e_{i}\rangle$ for every $i\in\{1,\ldots,d\}$. In fact $U|s_{n}\rangle$ need not belong to $H_{\mathcal{S}}^{\otimes\ell}$ for any $\ell\in\mathbb{N}$, but can in general be in a superposition of different lengths. (Hence the term indeterminate-length quantum codes.)

The Kraft-McMillan Inequality (Theorem 2.1) was initially extended to the quantum domain in [18] and subsequently in [13] and [5]. Before presenting (a slightly different) Quantum Kraft-McMillan Inequality, we will first introduce the length observable and quantum codes with length eigenstates. The length observable $\Lambda$ acting on $H_{\mathcal{A}}^{\oplus}$ is given by

[TABLE]

where $\Pi_{\ell}$ is the orthogonal projection onto the subspace $H_{\mathcal{A}}^{\otimes\ell}$ of $H_{\mathcal{A}}^{\oplus}$.

We say that a quantum code $U:H_{\mathcal{S}}\rightarrow H_{\mathcal{A}}^{\oplus}$ has length eigenstates if $U$ has the form

[TABLE]

for some orthonormal basis $\{|e_{i}\rangle\}_{i=1}^{d}$ of $H_{\mathcal{S}}$ and some sequence $\{|\psi_{i}\rangle\}_{i=1}^{d}\subseteq H_{\mathcal{A}}^{+}$ such that, for each $1\leq i\leq d$, $|\psi_{i}\rangle\in H_{\mathcal{A}}^{\otimes\ell_{i}}$ for some $\ell_{i}\in\mathbb{N}$.

Note that the $|\psi_{i}\rangle$’s are orthogonal due to $U$ being a linear isometry. It is easy to see that every c-q scheme is a quantum code with length eigenstates. Lastly, for each $\ell\in\mathbb{N}\cup\{0\}$, we will refer to the elements of the set $\{\psi_{i}:i\in\{1,\ldots,d\},\ \psi_{i}\in H_{\mathcal{A}}^{\otimes\ell}\}$ as the length $\ell$ eigenstates of $U$ and we will refer to $\{\ell_{i}\}_{i=1}^{d}$, where, for each $i=1,\ldots,d$, $\psi_{i}\in H_{\mathcal{A}}^{\otimes\ell_{i}}$, as the length eigenvalues of $U$.

Remark 2.8**.**

The quantum versions of the Kraft-McMillan Inequality proved in [18, Section IIC] and [13, Theorem 3.6] are more general than the same proved in [5, Theorem 1], although the formalisms are quite different in all three. Our version of the quantum Kraft-McMillan Inequality, presented below, is a generalization of [5, Theorem 1], but is not quite in the full generality of [18, Section IIC] (in the forward direction) because we only consider uniquely decodable codes (as opposed to the more general notion called condensable codes considered in [18]). However, our version does have a converse statement, similar to the classical Kraft-McMillan Inequality, which is missing from the aforementioned quantum versions.

Theorem 2.9**.**

(Quantum Kraft-McMillan Inequality) Any uniquely decodable quantum code $U$ with length eigenstates over a binary alphabet must satisfy the inequality

[TABLE]

Conversely, if $U:H_{\mathcal{S}}\rightarrow H_{\mathcal{A}}^{\oplus}$ is a linear isometry with length eigenstates satisfying the above inequality, then there exists a c-q scheme $\widetilde{U}$ with the same number of length $\ell$ eigenstates for each $\ell\in\mathbb{N}$.

The proof of Theorem 2.9 is presented in the Appendix.

We would like to find a quantum code which minimizes the amount of resources required. Unfortunately there are numerous ways to define the length of a codeword for an indeterminate-length quantum code (e.g. base length [6], exponential length [5, Definition 6], etc.). Here, we follow [5, Definition 3] and define the length of a codeword $|\omega\rangle$, which is a normalized vector in $H_{\mathcal{A}}^{\oplus}$ given by $|\omega\rangle=U|s\rangle$ for a unique symbol state $|s\rangle\in\{|s_{n}\rangle\}_{n=1}^{N}$, as the expectation with respect to the length observable in Equation (4). Explicitly, the length of a codeword $|\omega\rangle=U|s\rangle$ will be given by a function $\ell:H_{\mathcal{A}}^{\oplus}\rightarrow\mathbb{R}^{+}$, defined as follows:

[TABLE]

Whenever $U$ has length eigenstates and is given by Equation (5), we see that Equation (6) simplifies to

[TABLE]

where $\{\ell_{i}\}_{i=1}^{d}$ denotes the set of length eigenvalues of $U$.

Again we follow [5] and, for any ensemble $\mathcal{S}=\{p_{n},|s_{n}\rangle\}_{n=1}^{N}$, we define the ensemble state $\rho_{\mathcal{S}}$ of $\mathcal{S}$ by

[TABLE]

If $U$ is a quantum code on $H_{\mathcal{S}}$ define the average codeword length with respect to the ensemble $\mathcal{S}$ by

[TABLE]

We denote by $U_{\text{opt}}$ the optimal quantum code with length eigenstates for the ensemble $\mathcal{S}$ if

[TABLE]

where the second and third equality follow from Theorem 2.9, and the $\{\ell_{i}\}_{i=1}^{d}$ in the third equality denote the length eigenvalues of $U$. The existence of $U_{\text{opt}}$ follows from the existence of $C_{\text{opt}}$ in Equation (1) by the backward direction of Theorem 2.9. The optimal average codeword length for the ensemble $\mathcal{S}$ is given by

[TABLE]

It is shown in [5, Theorem 2] that the optimal c-q scheme (and hence optimal quantum code with length eigenstates by the converse of Theorem 2.9) is given by the classical Huffman codes. The bounds on $EL^{*}(\rho_{\mathcal{S}})$ in terms of the von-Neumann entropy follow immediately.

Theorem 2.10**.**

The minimum average codeword length for an ensemble $\mathcal{S}$ is bounded as follows,

[TABLE]

Proof.

See [5, Theorem 3]. ∎

Next, we wish to consider the optimal average codeword length per symbol for a collection of ensembles $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$, where $\mathcal{S}^{k}=\{p(n_{1},\cdots,n_{k}),|s_{1}s_{2}\cdots s_{k}\rangle\}_{n_{1},\ldots,n_{k}=1}^{N}$ and probabilities given by the pmf $p$ of a stochastic process $\mathbf{X}$. We will refer to such collections of ensembles as stochastic ensembles. Note that, by the definitions of a stochastic process, a stochastic ensemble $\mathcal{S}^{k}$ must be compatible in the following sense:

[TABLE]

for all $n_{1},\ldots,n_{k}\in\{1,\ldots,N\}\text{ and }k\in\mathbb{N}$. Notice that we allow for the possibility that preparations of the ensemble at each time be dependent upon previous preparations. If the preparations of the ensemble are independent and identically prepared copies of $\mathcal{S}=\{p_{n},|s_{n}\rangle\}_{n=1}^{N}$; i.e. the stochastic process $\mathbf{X}$ is made up of i.i.d. copies of a random variable $X$, then $p(n_{1},\cdots,n_{k})=p_{n_{1}}p_{n_{2}}\cdots p_{n_{k}}$ and $\rho_{\mathcal{S}^{k}}=\rho_{\mathcal{S}^{1}}^{\otimes k}$, where $\rho_{\mathcal{S}^{k}}=\sum_{n_{1},\ldots,n_{k}=1}^{N}p(n_{1},\ldots,n_{k})|s_{n_{1}}\cdots s_{n_{k}}\rangle\langle s_{n_{1}}\cdots s_{n_{k}}|$. For each $k\in\mathbb{N}$, let

[TABLE]

be the optimal average codeword length per symbol for the first $k$ symbols with respect to the ensemble $\mathcal{S}^{k}$, where $EL^{*}(\rho_{\mathcal{S}^{k}})$ is given by Equation (7). Notice that the optimal average codeword length per symbol is defined analogously to the classical case in Equation (2). Then, from Theorem 2.10, we have

[TABLE]

In the following section, we will relate the above quantities to the dynamical entropy of a quantum dynamical system.

3. An expression for the optimal quantum data compression rate using quantum dynamical entropy

3.1. A quantum dynamical system associated with a stationary Markov ensemble

In this article we consider stochastic processes $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ such that for some fixed $N<\infty$, each $X_{n}$ is a random variable with values in $\{1,\ldots,N\}$. If $\mathbf{X}$ is a stochastic process set $p_{\mathbf{X}}$ to be the pmf of $\mathbf{X}$, i.e. $p_{\mathbf{X}}$ is a probability measure on $\{0,\ldots,N\}^{\mathbb{N}}$, such that for every $k\in\mathbb{N}$ and $n_{1},\ldots,n_{k}\in\{1,\ldots,N\}$, $p_{\mathbf{X}}(n_{1},\ldots,n_{k})=\text{Pr}\,[X_{1}=n_{1},\ldots,X_{k}=n_{k}]$. We define the associated stochastic ensemble $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$ by setting $\mathcal{S}^{1}=\{p_{\mathbf{X}}(n),|s_{n}\rangle\}_{n=1}^{N}$ whose symbol states span $H_{\mathcal{S}}$ and $\mathcal{S}^{k}=\{p_{\mathbf{X}}(n_{1},\ldots,n_{k}),|s_{n_{1}}\cdots s_{n_{k}}\rangle\}_{n_{1},\ldots,n_{k}=1}^{N}$ whose symbol states span $H_{\mathcal{S}}^{\otimes k}$ for each $k\in\mathbb{N}$.

Recall that a stochastic process $\mathbf{X}$ is called stationary if the measure $p_{\mathbf{X}}$ is invariant with respect to the translation map, i.e. if for every $k\in\mathbb{N}$, $p_{\mathbf{X}}=p_{\mathbf{Y}^{(k)}}$ where the stochastic process $\mathbf{Y}^{(k)}=(Y^{(k)}_{n})_{n\in\mathbb{N}}$ is defined by $Y^{(k)}_{n}=X_{n+k}$ for every $n\in\mathbb{N}$. Obviously, if the stochastic process $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ is stationary then the random variables $X_{n}$ are identically distributed. The stochastic process $\mathbf{X}$ is called a Markov process if $\text{Pr}\,[X_{k+1}|X_{1},\ldots,X_{k}]=\text{Pr}\,[X_{k+1}|X_{k}]$ for every $k\in\mathbb{N}$. Hence a stochastic process $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ is a stationary Markov process if and only if the random variables $X_{n}$ are identically distributed and there exists a column stochastic matrix $(p_{i,j})_{i,j=1}^{N}$ such that for every $k\in\mathbb{N}$ and $n_{1},\ldots,n_{k},n_{k+1}\in\{1,\ldots,N\}$, we have

[TABLE]

Equivalently, a stochastic process $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ is a stationary Markov process if and only if the random variables $X_{n}$ are identically distributed and there exists a transition matrix $P=(p_{n,m})_{n,m=1}^{N}$ and an initial distribution $p=\{p_{n}\}_{n=1}^{N}$ such that the pmf of $\mathbf{X}$ is given by $p_{\mathbf{X}}(n_{1},\ldots,n_{k})=p_{n_{1}}\prod_{l=2}^{k}p_{n_{l},n_{l-1}}$, for each $k\in\mathbb{N}$ and $1\leq n_{1},\ldots,n_{k}\leq N$, and $p$ is invariant with respect to $P$; i.e. $Pp=p$; (the reason that $p$ is called “initial distribution” is because it coincides with the distribution of the random variable $X_{1}$). We will refer to a stochastic ensemble governed by a Markov process $\mathbf{X}$ as the Markov ensemble governed by $\mathbf{X}$. Whenever the Markov process is stationary we will refer to the Markov ensemble as being stationary.

Let $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$ be a stationary Markov ensemble governed by a stationary Markov process $\mathbf{X}$ having transition matrix $P=(p_{n,m})$ and initial distribution $p=\{p_{n}\}_{n=1}^{N}$. Setting $d=\dim(H_{\mathcal{S}})$, so that $d^{k}=\dim(H_{\mathcal{S}}^{\otimes k})$ for each $k\in\mathbb{N}$, the following sequence of ensemble states which represent this collection of ensembles is defined:

[TABLE]

and for each $k\in\mathbb{N}$ with $k\geq 2$,

[TABLE]

where we used the following notation.

Notation 3.1**.**

If $H$ is a Hilbert space then $S_{1}(H)$ will denote the space of trace-class operators on $H$. In the sequel we will frequently donote by $\mathcal{A}$ a unital $C^{*}$-algebra and $\Sigma(\mathcal{A})$ will denote the set of normal states on $\mathcal{A}$. Since we assume that $\mathcal{A}=B(H)$, we will identify each normal state, $\omega\in\Sigma(\mathcal{A})$ with its density operator $\rho\in S_{1}(H)$ through the identification $\omega(\cdot)=\operatorname{tr}\,(\rho\ \cdot)$.

We now define a quantum dynamical system associated with the above stationary Markov ensemble. Recall that a quantum dynamical system is a triplet $(\mathcal{A},\Theta,\rho)$ where $\mathcal{A}$ is a unital $C^{*}$-algebra, $\Theta:\mathcal{A}\to\mathcal{A}$ is a positive unital map, and $\rho$ is a density operator on $\mathcal{A}$. In some situations the map $\Theta$ is taken to be a $*$-automorphism, but we will not adopt this restriction here. The reason that we assume that $\Theta$ is positive and unital, is because we would like to have that the dual map $\Theta^{\dagger}$ maps the set of density operator of $\mathcal{A}$ (i.e. the set of positive unital functionals on $\mathcal{A}$) to itself. Throughout this paper we will, for simplicity, ignore the GNS construction and when we do not specify the $C^{*}$-algebra we will assume that it is equal to $\mathcal{A}=B(H)$ for some Hilbert space $H$.

The quantum dynamical system associated to the above stationary Markov ensemble is defined as follows: Let $\mathcal{A}=B(\mathbb{C}^{N})$, $\Theta:B(\mathbb{C}^{N})\to B(\mathbb{C}^{N})$ be defined by

[TABLE]

and let the density operator $\rho\in S_{1}(\mathbb{C}^{N})$ be defined by

[TABLE]

It is easy to verify that the map $\Theta$ is positive and unital. Indeed,

[TABLE]

thus $\Theta$ is unital. Also it is easy to verify that the dual map $\Theta^{\dagger}:S_{1}(\mathbb{C}^{N})\to S_{1}(\mathbb{C}^{N})$ is given by

[TABLE]

Note that throughout the article we use dagger to denote trace-class duality i.e.

[TABLE]

for all $m,n,k,\ell\in\{1,\ldots,N\}$, and star to denote Hermitian conjugate with respect to the underlying Hilbert space, $\mathbb{C}^{N}$. We thus obtain that the dual map $\Theta^{\dagger}$ is positive and trace-preserving.

Finally we recall that if $H$ is a Hilbert space, then an operational partition of unity on $H$ is a family $\gamma=(\gamma_{i})_{i=1}^{d}$ for some $d\in\mathbb{N}$, satisfying $\gamma_{i}\in B(H)$, for each $i$, and $\sum_{i=1}^{d}\gamma_{i}^{*}\gamma_{i}=\mathbbm{1}_{H}$. Let $\gamma=(\gamma_{i})_{i=1}^{d}$ be the operational partition of unity on the Hilbert space $\mathbb{C}^{N}$ associated with the above Markov ensemble, defined as follows:

[TABLE]

where $\{e_{i}\}_{i=1}^{d}$ is a fixed orthonormal basis of $H_{\mathcal{S}}$. Notice that

[TABLE]

where the second to last equality follows by Parseval’s identity. Hence $\gamma$ is indeed an operational partition of unity.

3.2. Quantum Dynamical Entropy via Quantum Markov Chains

In this subsection we recall the definition of quantum Markov chains (QMCs) and dynamical entropy thereon. Fix a quantum dynamical system $(\mathcal{A},\Theta,\rho)$ with $\mathcal{A}=B(H)$ for some Hilbert space $H$ and fix an operational partition of unity $\gamma=(\gamma_{i})_{i=1}^{d}$ on $H$. Following [21, Page 413] (see also [10, Equation 3.14]), we will consider the transition expectation

[TABLE]

given by the equation

[TABLE]

for some fixed orthonormal basis $\{e_{i}\}_{i=1}^{d}$ of $\mathbb{C}^{d}$. Further we define the transition expectation

[TABLE]

Its dual map

[TABLE]

(which is defined using trace duality), is usually called a lifting because it “lifts” states from $\mathcal{A}$ to $M_{d}\otimes\mathcal{A}$.

If $H$ is a Hilbert space, $\mathcal{A}$ is the von Neumann algebra $B(H)$ of all bounded operators on $H$, $\rho$ is a density operator on $H$ and, for some $d\in\mathbb{N}$, $\mathcal{E}:M_{d}\otimes\mathcal{A}\rightarrow\mathcal{A}$ is a transition expectation, then the pair $\{\rho,\mathcal{E}\}$ is called a quantum Markov chain (QMC). We will be specifically interested in QMCs whose transition expectation is given by Equation (14). Given a quantum Markov chain, we define the quantum Markov state $\psi$ on $M_{d}^{\otimes\mathbb{N}}$ by the equation

[TABLE]

for all $n\in\mathbb{N}$ and $a_{1},\ldots,a_{n}\in M_{d}$. Notice that the assumption that the transition expectation $\mathcal{E}$ is unital implies that $\psi$ is compatible in the sense that

[TABLE]

for all $n\in\mathbb{N}$ and $a_{1},\ldots,a_{n}\in M_{d}$. Moreover, it was shown in [2, Proposition 3.7] that the state $\psi$ on $M_{d}^{\otimes\mathbb{N}}$ indeed exists.

The joint correlations for $\psi$ are given by the density matrices $\rho_{n}\in M_{d}^{\otimes n}$ satisfying

[TABLE]

for all $n\in\mathbb{N}$ and $a_{1},\ldots,a_{n}\in M_{d}$.

Putting the above pieces together, if $\Theta:\mathcal{A}\rightarrow\mathcal{A}$ is a positive, unital map on the von Neumann algebra $\mathcal{A}=B(H)$, $\rho$ is a density operator on $H$, and $\gamma=(\gamma_{i})_{i=1}^{d}$ is an operational partition of unity of $H$, then the dynamical entropy of $(\mathcal{A},\Theta,\rho)$ with respect to $\gamma$ is given by

[TABLE]

where $S(\cdot)$ is the von-Neumann entropy and the transition expectation is given by Equation (14). Further, given a subalgebra $\mathcal{B}$ of $\mathcal{A}$, the dynamical entropy of $(\mathcal{A},\Theta,\rho)$ with respect to $\mathcal{B}$ is given by

[TABLE]

Remark 3.2**.**

The dynamical entropy above is the generalized AF dynamical entropy as defined by the authors of [10]. The description we give is very similar to that of the AF dynamical entropy given by Tuyls in [21]; however, we do not restrict ourselves to $*$-automorphisms as does the standard construction of AF dynamical entropy.

3.3. Computation of the quantum dynamical entropy of the quantum dynamical system defined in Subsection 3.1

Let $(\mathcal{A},\Theta,\rho)$ be the quantum dynamical system defined by (9) and (10), and let $\gamma$ be the operational partition of unity defined by (12). In this subsection, we will use the definitions given in Subsection 3.2 in order to compute the quantum dynamical entropy $h(\Theta,\rho,\gamma)$ and give its interpretation as the optimal compression rate of the quantum Markov ensemble that we coinsider.

First we define vectors

[TABLE]

(which are orthonormal even though the vectors $(|s_{n}\rangle)_{n=1}^{N}\subseteq H_{\mathcal{S}}$ are not necessarily mutually orthogonal), and the state

[TABLE]

Before proceeding with the construction of the quantum Markov chain, we give a technical lemma which will be helpful later.

Lemma 3.3**.**

Let $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$ be a stationary Markov ensemble, with symbol states $\{|s_{n}\rangle\}_{n=1}^{N}$, which is governed by a stationary Markov process $\mathbf{X}$ with transition matrix $P=(p_{n,m})$. Let $\Theta$, $\gamma$, $\mathcal{E}_{\gamma}$ and $\mathcal{E}_{\gamma,\Theta}$ be defined as above. Then the lifting $\mathcal{E}_{\gamma,\Theta}^{\dagger}:S_{1}(\mathbb{C}^{N})\rightarrow S_{1}(H_{\mathcal{S}})\otimes S_{1}(\mathbb{C}^{N})$ acts on the diagonal states of $S_{1}(\mathbb{C}^{N})$ in the following way.

[TABLE]

for each $|n\rangle$ in the orthonormal basis of $\mathbb{C}^{N}$. Moreover,

[TABLE]

where $\rho$ and $\rho^{\prime}$ are given by Equations (10) and (18), respectively.

The proof of Lemma 3.3 can be found in the Appendix.

Next, we will consider the quantum Markov state $\psi$ given by the chain $\{\rho,\mathcal{E}_{\gamma,\Theta}\}$, where $\rho$ is given in Equation (10) and $\mathcal{E}_{\gamma,\Theta}$ is as in Equation (14). Then, for each $k\in\mathbb{N}$ and $a_{1},\ldots,a_{k}\in B(H_{\mathcal{S}})=M_{d}$, we have

[TABLE]

where the “moreover” part of Lemma 3.3 was used in the $3^{\text{rd}}$ equality, the fact $\operatorname{tr}\,(A\otimes B)=\operatorname{tr}\,(A)\operatorname{tr}\,(B)$ was used in the $4^{\text{th}}$ equality and Lemma 3.3 was used in the $5^{\text{th}}$ equality.

Thus, for each $k\in\mathbb{N}$, the density matrix $\rho_{k}$ which is defined by Equation (16) is given by

[TABLE]

Therefore,

[TABLE]

where the first equality holds by the definition of the dynamical entropy in Equation (17) and the last equality follows from Equation (8). We have proved the following result.

Theorem 3.4**.**

Given any stationary Markov ensemble $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$, the optimal average codeword length per symbol (via lossless coding) converges to the dynamical entropy of the above-described quantum dynamical system $(B(\mathbb{C}^{N})),\Theta,\rho)$ with respect to the operational partition of unity $\gamma$ defined in Equation (12) in the following sense:

[TABLE]

We recover the result of Schumacher [17] and Bellomo et. al [5] which states that the optimal codeword length per symbol for an i.i.d. prepared ensemble, $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$, (via asymptotically lossless coding) is equal to the von Neumann entropy of the initial ensemble state, $\rho_{\mathcal{S}^{1}}$:

Corollary 3.5**.**

Given a Markov process $\mathbf{X}$ made up of i.i.d. copies of a random variable $X$, the stationary Markov ensemble $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$ governed by $\mathbf{X}$ has optimal codeword length per symbol (via lossless coding) given by

[TABLE]

Proof.

First notice that $\mathbf{X}$ is governed by the transition matrix $P=(p_{n,m})_{n,m=1}^{N}$ such that $p_{n,m}=p_{n}$, for every $1\leq n,m\leq N$, where $p=(p_{n})_{n=1}^{N}$ is the initial distribution of $\mathbf{X}$. Therefore

[TABLE]

Using the construction from above and Equation (19), we have that

[TABLE]

where the last inequality follows by additivity of von Neumann entropy (see e.g. [22, Equation 2.8]). Therefore, by Theorem 3.4, we have

[TABLE]

∎

Next we turn to a similar representation for general stochastic ensembles. We chose to present the case of the stationary Markov ensemble separately since the construction is simpler than in the general case.

3.4. A quantum dynamical system associated with a general stochastic ensemble

Consider a stochastic process $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ with values in $\{1,\ldots,N\}$ for some $N<\infty$ and with pmf $p$ i.e. for any $k\in\mathbb{N}$ and any $(n_{1},\ldots,n_{k})\in\{1,\ldots,N\}^{k}$ we have $p(n_{1},\ldots,n_{k})=\text{Pr}\,[X_{1}=n_{1},\ldots,X_{k}=n_{k}]$. Define the associated stochastic ensemble $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$ by $\mathcal{S}^{1}=\{p(n),|s_{n}\rangle\}_{n=1}^{N}$ whose symbol states span $H_{\mathcal{S}}$ and $\mathcal{S}^{k}=\{p(n_{1},\ldots,n_{k}),|s_{n_{1}}\cdots s_{n_{k}}\rangle\}_{n_{1},\ldots,n_{k}=1}^{N}$ whose symbol states span $H_{\mathcal{S}}^{\otimes k}$ for each $k\in\mathbb{N}$. Again, setting $d=\dim(H_{\mathcal{S}})$, so that $d^{k}=\dim(H_{\mathcal{S}}^{\otimes k})$ for each $k\in\mathbb{N}$, we define the following sequence of ensemble states which represents this stochastic ensemble:

[TABLE]

and, for each $k\in\mathbb{N}$ with $k\geq 2$, define $\rho_{S^{k}}\in M_{d}^{\otimes k}=S_{1}(H_{\mathcal{S}}^{\otimes k})$ by

[TABLE]

We define a quantum dynamical system associated to the above quantum ensemble as follows: Let $H=(\mathbb{C}^{N})^{\oplus}=\oplus_{n=0}^{\infty}(\mathbb{C}^{N})^{\otimes n}$ be the free Fock space of $\mathbb{C}^{N}$. Recall that $(\mathbb{C}^{N})^{\otimes 0}=\mathbb{C}$ and we denote by $|\emptyset\rangle$ the vector $1\in(\mathbb{C}^{N})^{\otimes 0}$. We denote $\{|n\rangle:n\in\{1,\ldots,N\}\}$ the standard orthonormal basis of $\mathbb{C}^{N}$ and

[TABLE]

the standard orthonormal basis of $H$. Before proceeding further we introduce two useful notations on the standard orthonormal basis of $H$ which will be used later. If $\overline{n}=(n_{1},\ldots,n_{k})\in\{1,\ldots,N\}^{k}$ for some $k\in\mathbb{N}$, then we set

[TABLE]

Set $\mathcal{A}$ to denote $C^{*}$-subalgebra of $B(H)$ generated by the identity operator and the rank-one operators of the form $|\bar{n}\rangle\langle\bar{m}|$ where $\bar{n},\bar{m}\in\{1,\ldots,N\}^{+}$. Define a unital map $\Theta:\mathcal{A}\to\mathcal{A}$ by

[TABLE]

[TABLE]

[TABLE]

It is easy to see that the map $\Theta$ is positive and (by definition) unital. Finally we define a state

[TABLE]

on $\mathcal{A}$ and consider the dynamical system $(\mathcal{A},\Theta,\rho)$.

We also define an operational partition of unity of the Hilbert space $H$ which is associated to the above quantum ensemble. Let $\{e_{i}\}_{i=1}^{d}$ be a fixed orthonormal basis of $H_{\mathcal{S}}$, and let $\gamma=(\gamma_{i})_{i=1}^{d}$ be the operational partition of unity defined by by

[TABLE]

for each $i\in\{1,\ldots,d\}$. Notice that

[TABLE]

for $\bar{n}\in\cup_{k\in\mathbb{N}}\{1,\ldots,N\}^{k}$. Similarly,

[TABLE]

and thus $\gamma$ is indeed an operational partition of unity on $H$. In order to unify the last three displayed formulas, we define $\text{final}\,(\emptyset)=1$ and thus we can write

[TABLE]

Let $\mathcal{E}_{\gamma}$ and $\mathcal{E}_{\gamma,\Theta}$ be the transition expectation maps from $M_{d}\otimes\mathcal{A}$ to $\mathcal{A}$ by Equation (13) and (14), respectively.

Before stating the next result we introduce some notation.

Notation 3.6**.**

For each $k\in\mathbb{N}$ and $\bar{n}\in\{1,\ldots,N\}^{k}$ we set

[TABLE]

Also for $\bar{n}=(n_{1},\ldots,n_{k})\in\{1,\ldots,N\}^{k}$ and $\ell\in\{1,\ldots,N\}$ we set

[TABLE]

We now state a technical lemma which will be used in the proof of the main result.

Lemma 3.7**.**

Let $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$ be a stochastic ensemble with symbol states $\{|s_{n}\rangle\}_{n=1}^{N}$ which is governed by a stochastic process $\mathbf{X}$ with pmf $p$. Let $H$, $\mathcal{A}$, $\Theta$, $\rho$, $\gamma$, $\mathcal{E}_{\gamma}$ and $\mathcal{E}_{\gamma,\Theta}$ be defined as above. Then the lifting $\mathcal{E}_{\gamma,\Theta}^{\dagger}:\Sigma(\mathcal{A})\rightarrow M_{d}\otimes\Sigma(\mathcal{A})$ acts on the diagonal states of $S_{1}(H)$ in the following way:

[TABLE]

for each $|\bar{n}\rangle$ in the standard orthonormal basis of $H$ where we adopt the convention $p(k|\emptyset):=p(k)=\text{Pr}\,[X_{1}=k]$ for $k\in\{1,\ldots,N\}$. Moreover,

[TABLE]

The proof of Lemma 3.7 can be found in the Appendix.

Next, we will consider the quantum Markov state $\psi$ given by the chain $\{\rho,\mathcal{E}_{\gamma,\Theta}\}$. For each $k\in\mathbb{N}$ and $a_{1},\ldots,a_{k}\in B(H_{\mathcal{S}})=M_{d}$, we have

[TABLE]

where the “moreover” part of Lemma 3.7 was used in the $3^{\text{rd}}$ equality, the fact $\operatorname{tr}\,(A\otimes B)=\operatorname{tr}\,(A)\operatorname{tr}\,(B)$ was used in the $4^{\text{th}}$ equality and Lemma 3.7 was used in the $5^{\text{th}}$ equality.

Thus, for each $k\in\mathbb{N}$, the density matrix $\rho_{k}$ which is defined in Equation (16) is given by

[TABLE]

Therefore,

[TABLE]

where the first equality holds by the definition of the dynamical entropy in Equation (17) and the last equality follows from Equation (8). We have proved the following theorem.

Theorem 3.8**.**

Given any stochastic ensemble $\{\mathcal{S}^{k}\}_{k=1}^{\infty}$, the optimal average codeword length per symbol (via lossless coding) converges to the dynamical entropy of the above-described quantum dynamical system $(\mathcal{A},\Theta,\rho)$ with respect to the operational partition of unity $\gamma$ defined by Equation (20) in the following sense:

[TABLE]

It should be noted that Theorem 3.4 can be considered a corollary of Theorem 3.8. However, we have presented it separately since the construction is simpler in the case of Markov ensembles.

3.5. Examples

Examples of quantum sources that produce not-necessarily statistically independent quantum symbols have been considered in the literature. In [9] examples of quantum sources that produce statistically independent symbols (called “Bernoulli sources”) as well as quantum sources producing not-necessarily statistically independent quantum symbols are considered. In [6] the authors consider quantum Morse codes as an example of quantum communication since quantum data compression can be viewed as a special case of noiseless quantum communication. In communications, either classical or quantum, the assumption of statistical independence of the symbols to be communicated, gives a serious restriction to the content of information which is communicated. Thus the need of considering quantum sources emitting not-necessarily statistical independent quantum symbols, naturally arises.

We have already shown in Corollary 3.5 that we can recover the result of [17] and [5] which states that the optimal codeword length per symbol for an i.i.d. prepared ensemble (i.e. Bernoulli sources) is equal to the von Neumann entropy of the initial ensemble state. First we illustrate that we can recover Theorem 2.5 from Theorem 3.8.

Example 3.9** (Classical-Quantum Codes).**

Let $S=\{n\}_{n=1}^{d}$ be a classical symbol set of cardinality $d$ for some $d\in\mathbb{N}$, $C:S\rightarrow A^{+}$ be a uniquely decodable code into strings from the binary alphabet $A$, and $\mathbf{X}=(X_{n})_{n=1}^{\infty}$ be a stochastic process governing the frequency of symbols from the symbol set $S$. Let $H_{\mathcal{S}}$ be a $d$-dimensional Hilbert space spanned by an orthonormal basis $\{|s_{n}\rangle\}_{n=1}^{d}$ and define the stochastic ensemble as usual by $\mathcal{S}^{k}=\{p(n_{1},\ldots,n_{k}),|s_{n_{1}}\cdots s_{n_{k}}\rangle\}_{n_{1},\ldots,n_{k}=1}^{d}$, where $p$ denotes the pmf of $\mathbf{X}$. Then since the $|s_{n}\rangle$’s are orthonormal it is easy to see that the ensemble states $\rho_{\mathcal{S}^{k}}$ are diagonal, for each $k\in\mathbb{N}$. Hence the optimal average codeword length per symbol for the stochastic ensemble, given by $h(\mathcal{A},\Theta,\rho)$ in Theorem 3.8, is exactly equal to the entropy rate of the stochastic process (see Theorem 2.5).

Next we illustrate here the usefulness of Theorem 3.4 on non-Bernoulli sources with two simple examples. For each of the two examples, the Hilbert space $H_{\mathcal{S}}$ has dimension $d=2$ and an orthonormal basis $\{|e_{i}\rangle\}_{i=1}^{2}$.

Example 3.10**.**

For the second example consider the normalized non-orthogonal symbols $|s_{1}\rangle=|e_{1}\rangle$, $|s_{2}\rangle=|e_{2}\rangle$, $|s_{3}\rangle=\frac{1}{\sqrt{2}}(|e_{1}\rangle+|e_{2}\rangle)$, (i.e. the Bell state $|+\rangle$), and $|s_{4}\rangle=\frac{1}{\sqrt{2}}(|e_{1}\rangle-|e_{2}\rangle)$, (i.e. the Bell state $|-\rangle$), which span $H_{\mathcal{S}}=\mathbb{C}^{2}$ (i.e. we consider $N=4$ in the setting described in Subsection 2.2). Consider the transition matrix

[TABLE]

where $p_{i,j}$ represents the conditional probability that the quantum source emits $|s_{i}\rangle$ right after it emits $|s_{j}\rangle$. A (non-unique) fixed probability distribution of $P$ is equal to the column vector $(\frac{1}{4}\quad\frac{1}{4}\quad\frac{1}{4}\quad\frac{1}{4})^{T}$. Consider the quantum dynamical system $(\mathcal{A},\Theta,\rho)$ where $\mathcal{A}=B(\mathbb{C}^{4})$, $\Theta:\mathcal{A}\to\mathcal{A}$ is given by

[TABLE]

i.e.

[TABLE]

and

[TABLE]

Consider the operational partition of unity $\gamma=(\gamma_{i})_{i=1}^{2}$ for $\mathbb{C}^{4}$ given as in Equation (12). Theorem 3.4 states that the optimal average codeword length per symbol via lossless coding is equal to the the dynamical entropy of $\Theta$ with respect to the partition $\gamma$ when measured using the state $\rho$, i.e. $h(\Theta,\rho,\gamma)$. We can compute the joint correlations $(\rho_{n})_{n=1}^{\infty}$ of this dynamical system using Equation (19) to see that

[TABLE]

*and in general *

[TABLE]

It is easy to verify that

[TABLE]

by applying these sums to bases of $\mathbb{C}^{2^{k}}$ that are formed by taking the tensor products of $|s_{i}\rangle$’s. Thus $\rho_{k}=\frac{1}{2^{k}}\mathbbm{1}_{\mathbb{C}^{2^{k}}}$ and hence $S(\rho_{k})=k$, for each $k\in\mathbb{N}$. Therefore by Theorem 3.4 we obtain that the optimal average compression rate per symbol for the above quantum ensemble is equal to 1 qubit.

Example 3.11**.**

Consider the normalized non-orthogonal symbol states $|s_{1}\rangle=|e_{1}\rangle$, $|s_{2}\rangle=-\frac{1}{2}|e_{1}\rangle+\frac{\sqrt{3}}{2}|e_{2}\rangle$ and $|s_{3}\rangle=-\frac{1}{2}|e_{1}\rangle-\frac{\sqrt{3}}{2}|e_{2}\rangle$ which span $H_{\mathcal{S}}$ (i.e. we consider $N=3$ in the setting described in Subsection 2.2). Consider the transition matrix

[TABLE]

where $p_{i,j}$ represents the conditional probability that the quantum source emits $|s_{i}\rangle$ right after it emits $|s_{j}\rangle$. The unique fixed probability distribution of $P$ is equal to the column vector $(\frac{1}{3}\quad\frac{1}{3}\quad\frac{1}{3})^{T}$. Consider the quantum dynamical system $(\mathcal{A},\Theta,\rho)$ where $\mathcal{A}=B(\mathbb{C}^{3})$, $\Theta:\mathcal{A}\to\mathcal{A}$ is given by

[TABLE]

i.e.

[TABLE]

and

[TABLE]

Consider the operational partition of unity $\gamma=(\gamma_{i})_{i=1}^{2}$ for $\mathbb{C}^{3}$ given by

[TABLE]

and

[TABLE]

Theorem 3.4 states that the optimal average codeword length per symbol via lossless coding is equal to the the dynamical entropy of $\Theta$ with respect to the partition $\gamma$ when measured using the state $\rho$, i.e. $h(\Theta,\rho,\gamma)$. We can compute the joint correlations $(\rho_{n})_{n=1}^{\infty}$ of this dynamical system using Equation (19) to see that

[TABLE]

*and in general *

[TABLE]

where

[TABLE]

and we adopt the convention that the mod 3 function takes values in the set $\{1,2,3\}$. Using Matlab we can obtain the following approximate values of the von Neumann entropies of the above matrices:

[TABLE]

The above decreasing numbers indicate that the optimal average compression rate per symbol for the above quantum stochastic ensemble is strictly less than 1 qubit.

4. Concluding Remarks

In this paper, we developed further the theory of quantum data compression for indeterminate length quantum codes, building on the previous work of Schumacher and Westmoreland [18] and Bellomo, et. al [5]. We presented the quantum Kraft inequality with an additional converse statement which was not present in previous works; this additional converse statement makes the statement of the quantum Kraft inequality more reminiscent of its classical counterpart. We also introduce the notion of stochastic ensembles and, in particular, stationary Markov ensembles which, to the best of our knowledge, have not been considered elsewhere. The main contributions of this work are Theorems 3.4 and 3.8 which give a dynamical entropy interpretation of the optimal compression rate for stationary Markov and identically distributed stochastic ensembles, respectively, extending the results of Schumacher [17] and Bellomo et. al [5] where the quantum symbol states to be encoded were prepared in an i.i.d. way. In doing so, we give a quantum Markov chain representation of a particular open quantum random walk. An interesting direction for future study is the development of quantum data compression on the symmetric Fock space which is commonly used to model photons. We hope to develop this theory further in future work.

5. Appendix: Proofs of auxiliary results

Proof of Theorem 2.9:.

For the forward direction we adapt the proof of [18, Subsection II.C.] to our formalism. Let $U$ be a uniquely decodable quantum code with length eigenstates of the form

[TABLE]

and let $\{\ell_{i}\}_{i=1}^{d}$ be the length eigenvalues of $U$. For each $n,N\in\mathbb{N}$, let

[TABLE]

be the collection of length $N$ strings consisting of $n$-many codewords and let

[TABLE]

be the number of length $\ell$ eigenstates of $U$, for each $\ell\in\mathbb{N}$. Then, by the unique decodability of $U$, each element of $C_{n}^{N}$ has a unique representation as a string of $n$ codewords and the elements of $C_{n}^{N}$ are pairwise orthogonal, and hence we have

[TABLE]

Thus

[TABLE]

Set $\displaystyle{\ell_{\text{max}}=\max_{1\leq i\leq d}\{\ell_{i}\}}$ so that $N\leq n\ell_{\text{max}}$. Summing the above inequality over $N$ we obtain

[TABLE]

Notice that the left-hand side of this inequality is exponential whereas the right-hand side is linear. This implies that the left-hand side is bounded above by 1. Hence we must have that

[TABLE]

Notice that the inequality in Equation (21) is simply a restatement of the classical Kraft-McMillan inequality.

Conversely, suppose that $U$ is a linear isometry with length eigenstates satisfying the quantum Kraft-McMillan Inequality, and define $\{\ell_{i}\}_{i=1}^{d}$, $\ell_{\text{max}}$ and $\{d_{\ell}\}_{\ell=1}^{\ell_{\text{max}}}$ as above. Then

[TABLE]

and hence the classical Kraft-McMillan inequality is also valid. Thus, by the converse of the classical Kraft-McMillan Theorem, one can find a classical uniquely decodable code $C$ which has exactly $d_{\ell}$-many codewords of length $\ell$, for each $\ell\in\mathbb{N}$. The c-q scheme $\widetilde{U}$ constructed from this classical code $C$ has the desired properties. ∎

Proof of Lemma 3.3:.

Since $\mathcal{E}_{\gamma,\Theta}=\Theta\circ\mathcal{E}_{\gamma}$ we have that $\mathcal{E}_{\gamma,\Theta}^{\dagger}=\mathcal{E}_{\gamma}^{\dagger}\circ\Theta^{\dagger}$.

Next we consider the lifting $\mathcal{E}_{\gamma}^{\dagger}:S_{1}(\mathbb{C}^{N})\rightarrow S_{1}(H_{C})\otimes S_{1}(\mathbb{C}^{N})$ which we claim is given by the formula

[TABLE]

where we have identified $S_{1}(H_{\mathcal{S}})$ with $M_{d}$ given the matrix representation with respect to the fixed orthonormal basis $\{|e_{i}\rangle\}_{i=1}^{d}$ used in Equations (13) and (12). Indeed, for $[a_{i,j}]_{i,j=1}^{d}\in B(H_{\mathcal{S}})\otimes B(\mathbb{C}^{N})$ and $\sigma\in S_{1}(\mathbb{C}^{N})$, we have

[TABLE]

which proves the validity of Equation (22). Then, for each $|m\rangle\langle m|\in S_{1}(\mathbb{C}^{N})$, we have

[TABLE]

Combining Equations (9) and (23), for each $|n\rangle\langle n|\in S_{1}(\mathbb{C}^{N})$, we have

[TABLE]

For the moreover statement, we have

[TABLE]

∎

Proof of Lemma 3.7:.

It is easy to see that for each $\bar{m},\bar{n}\in\{1,\ldots,N\}^{+}$ (i.e.  $|\bar{m}\rangle,|\bar{n}\rangle$ belong in the standard orthonormal basis of $H$), we have

[TABLE]

Next we consider the lifting $\mathcal{E}_{\gamma}^{\dagger}:S_{1}(H)\rightarrow S_{1}(H_{\mathcal{S}})\otimes S_{1}(H)$ which (by Equation (22)) is given by the formula

[TABLE]

where we have identified $S_{1}(H_{\mathcal{S}})$ with $M_{d}$ given the matrix representation with respect to the fixed orthonormal basis $\{|e_{i}\rangle\}_{i=1}^{d}$ of the Hilbert space $H_{\mathcal{S}}$. Then, for each $\bar{n}\in\{1,\ldots,N\}^{+}$, we have

[TABLE]

where we used Equation (20) in the second equality.

Combining Equations (25) and (26), for each $\bar{n}\in\{1,\ldots,N\}^{+}$, we have

[TABLE]

For the moreover statement, we have

[TABLE]

where we again used the convention that $p(k|\emptyset)=p(k)$, for all $k\in\{1,\ldots,N\}$, in the last equality. ∎