Quantum Rate-Distortion Coding of Relevant Information

TL;DR

This paper extends the classical Information Bottleneck Method to quantum data, deriving a quantum rate-distortion function using entanglement-assisted protocols and addressing unique quantum formalism challenges.

Contribution

It introduces a quantum version of the Information Bottleneck Method for data compression, utilizing a channel communication framework and entanglement assistance.

Findings

Derived quantum rate-distortion function under entanglement assistance

Characterized optimal quantum channels for given input states

Discussed and addressed formalism differences between quantum and classical probability

Abstract

Rate-distortion theory provides bounds for compressing data produced by an information source to a specified encoding rate that is strictly less than the source's entropy. This necessarily entails some loss, or distortion, between the original source data and the best approximation after decompression. The so-called Information Bottleneck Method is designed to compress only `relevant' information. Which information is relevant is determined by the correlation between the data being compressed and a variable of interest, so-called side information. In this paper, an Information Bottleneck Method is introduced for the compression of quantum data. The channel communication picture is used for compression and decompression. The rate of compression is derived using an entanglement assisted protocol with classical communication, and under an unproved conjecture that the rate function is convex in the distortion parameter. The optimum channel achieving this rate for a given input state is characterised. The conceptual difficulties arising due to differences in the mathematical formalism between quantum and classical probability theory are discussed and solutions are presented.

Equations

$$R_{\text{c}ls}(C)=\underset{P(\tilde{X}|X):I(\tilde{X};Y)\ge C}{\min }I(X;\tilde{X}),$$

$${\mathcal{L}}_{cls}=I(X;\tilde{X})-\beta I(\tilde{X};Y)-\sum _{x,\tilde{x}}\lambda (x)P(\tilde{x}|x),$$

$$P(\tilde{x}|x)=P(\tilde{x})\frac{e^{-\beta D(P(Y|x)||P(Y|\tilde{x}))}}{Z}.$$

$${\mathcal{F}}_n^{(i)}({\xi }_X):=$$

$${\mathrm{Tr}}_{{\tilde{X}}_1,...,{\tilde{X}}_{i-1},{\tilde{X}}_{i+1},...,{\tilde{X}}_n}[{\mathcal{F}}_n({\rho }_X^{\otimes (i-1)}\otimes {\xi }_X\otimes {\rho }_X^{\otimes (n-i)})].$$

$${\tilde{\sigma }}_{{\tilde{X}}_i{Y}_i}$$

$${\tilde{\sigma }}_{{\tilde{X}}_i}={\mathcal{F}}_n^{(i)}({\rho }_X),{\tilde{\sigma }}_{Y_i}={\rho }_Y.$$

$$I_i(\tilde{X};Y)$$

$${\bar{I}}_n(\tilde{X};Y):=\frac{1}{n}\sum _{i=1}^n{I}_i(\tilde{X};Y).$$

$$\underset{n\to \infty }{\lim }{\bar{I}}_n(\tilde{X};Y)>J.$$

$$R(J):=\inf \{r:(r,J)\text{mbox}isachievable\}.$$

$$R(J)=\underset{{\mathcal{N}}_{X\to \tilde{X}}:I(\tilde{X};Y{)}_{\tilde{\rho }}\ge J}{\min }I(X^{\prime };\tilde{X}{)}_{\tilde{\tau }},$$

$${\tilde{\tau }}_{X^{\prime }\tilde{X}}:=({\mathcal{I}}_{X^{\prime }}\otimes {\mathcal{N}}_{X\to \tilde{X}})({\tau }_{X^{\prime }X}),$$

$${\tilde{\rho }}_{\tilde{X}Y}:=({\mathcal{N}}_{X\to \tilde{X}}\otimes {\mathcal{I}}_Y)({\rho }_{XY})$$

$$M(J):=\underset{{\mathcal{N}}_{X\to \tilde{X}}:I(\tilde{X};Y{)}_{\tilde{\rho }}\ge J}{\min }I(X^{\prime };\tilde{X}{)}_{\tilde{\tau }}.$$

$$nr$$

$$\ge S(W|T_{\tilde{X}})$$

$$\ge S(W|T_{\tilde{X}})-S(W|X^{\prime n}{T}_{\tilde{X}})$$

$$=I(W;X^{\prime n}|T_{\tilde{X}})$$

$$=I(W;X^{\prime n}|T_{\tilde{X}})+I(X^{\prime n};T_{\tilde{X}})$$

$$=I(W{T}_{\tilde{X}};X^{\prime n})$$

$$\ge I({\tilde{X}}^n;X^{\prime n})$$

$$\ge \sum _{i=1}^{n}I({\tilde{X}}_i;X_i^{\prime })$$

$$\ge \sum _{i=1}^{n}M(I_i(\tilde{X};Y))$$

$$=n\sum _{i=1}^n\frac{1}{n}M(I_i(\tilde{X};Y))$$

$$\ge nM({\bar{I}}_n(\tilde{X};Y))$$

$$\ge nM(J).$$

$$\Vert {\tilde{\sigma }}_{X^{\prime n}{\tilde{X}}^n}-{\tilde{\tau }}_{X^{\prime }\tilde{X}}^{\otimes n}{\Vert }_1\le ϵ,$$

$${\tilde{\sigma }}_{X^{\prime n}{\tilde{X}}^n}:={\mathcal{I}}_{X^{\prime n}}\otimes {\mathcal{F}}_n({\tau }_{X^{\prime }X}^{\otimes n}).$$

$$\forall \delta >0:M(J-\delta )\le R(J-\delta )\le M(J).$$

$$\underset{\delta \searrow 0}{\lim }R(J-\delta )=M(J).$$

$$\displaystyle\Psi_{X^{\prime}\tilde{X}}:=\big{(}\mathcal{I}_{X^{\prime}}\otimes\mathcal{N}_{X\to{\tilde{X}}}\big{)}(\tilde{\Phi}_{X^{\prime}X}),$$

$$\mathcal{L}$$

$${\tilde{\tau }}_{X^{\prime }\tilde{X}}$$

$$=({\rho }_{X^{\prime }}^X\otimes I_{\tilde{X}}{)}^{1/2}{\Psi }_{X^{\prime }\tilde{X}}^{t_{X^{\prime }}}({\rho }_{X^{\prime }}^X\otimes I_{\tilde{X}}{)}^{1/2},$$

$${\tilde{\rho }}_{\tilde{X}Y}:=({\mathcal{N}}_{X\to \tilde{X}}\otimes {\mathcal{I}}_y)({\rho }_{XY})$$

$${\tilde{\rho }}_{\tilde{X}Y}=$$

Full text

I Introduction

One of the most central results in classical information theory is Shannon’s data compression theorem [13] which gives a fundamental limit on lossless compressibility of data. Due to statistical redundancies, data can be compressed at a rate bounded below by the source entropy, such that after decompression the full information is recovered without loss. Rate-distortion theory (RDT) is the branch of information theory that compresses the data produced by an information source down to a specified encoding rate that is strictly less than the source entropy [14]. This necessarily entails some loss, or distortion, between the original source data and the best approximation after decompression, according to some distortion measure. RDT is frequently used in multimedia data compression where a large amount of data can be discarded without any noticeable change to a listener or viewer.

Whilst RDT is an important and widely used concept in information theory, there are cases where only part of the information in the data to be compressed is relevant. For instance in speech signal processing one might be interested only in information about the spoken words in audio data. The Information Bottleneck Method (IBM), introduced by Tishby et al., achieves a lossy compression rate even lower than the rate given by RDT by keeping only ‘relevant’ information [16]. Which information is relevant is determined by the correlation between the data being compressed and a variable of interest. The information to be recovered after decoding is only the relevant part of the source data. For example, one might have access to the transcript of an audio recording which has an entropy by orders of magnitude lower than the original audio data. The transcript here can be used as side information to compress the audio data further than what can be achieved by RDT, without increasing the distortion of the relevant information.

Loss of information in the context of a compression-decompression scheme is mathematically equivalent to transmission of information through a noisy channel. Rather than characterising the information lost by encoding, one characterises the information lost during transmission. The Information Bottleneck Method is formulated as a communication problem with the relevant variable acting as side information. Iterative algorithms to compute the optimum channel achieving the task are also provided in [16].

In [12], a first quantum extension of the Information Bottleneck Method was developed and applied to predictive filtering. In this paper we extend the Information Bottleneck Method to the quantum case by considering the transmission of quantum information through a quantum channel with side information. We derive the compression rate for an entanglement-assisted protocol with classical communication for our quantum extension of the IBM. Our derivation relies on the conjecture that the rate function is convex in distortion. The optimum quantum channel that can achieve this rate is also characterised.

A quantum extension to RDT was introduced by Barnum [3]. However, the results were unsatisfactory since the bound on the rate was given in terms of coherent information which can be negative. The results were improved and a complete characterisation of quantum channels achieving rate-distortion coding in a number of settings was given by Datta et al. [8]. Various settings of quantum RDT in the presence of auxiliary information were discussed in the work of Wilde et al. [18]. However, the specific question of transmitting relevant information asked in the IBM with its choice of distance measure and the specifics of the optimisation problem have not been considered yet.

The setting of the classical IBM is as follows. A correlated pair of random variables $X$ and $Y$ is given with a joint probability distribution $P(X,Y)$. The task is to find the optimum channel with input $X$ and output $\tilde{X}$ such that $\tilde{X}$ retains a fixed amount of correlation, $C$, with variable $Y$. The amount of correlation is quantified by the Shannon mutual information $I(\tilde{X};Y):=H(\tilde{X})+H(Y)-H(\tilde{X}Y)$, where $H(\cdot)$ is the Shannon entropy. For a successful completion of the IBM task it is required that $I(\tilde{X};Y)\geq C$. Representing the channel by the conditional probability distribution $P(\tilde{X}|X)$, one can show that the classical rate of compression for a given minimum amount of correlation $C$, $R_{\text{c}ls}(C)$, is given by

[TABLE]

as it was first proposed in [16] and proved in [11].

The IBM, however, is concerned not with an output distribution close to the input distribution but with an output characterised by its information about some other variable $Y$. The task of the IBM is to find the lowest value of $I(X;\tilde{X})$ such that $I(\tilde{X};Y)$ is still above some given threshold. The value of $I(X;\tilde{X})$ can be reinterpreted as a communication rate, namely the number of transmitted bits needed to specify an element of $\tilde{X}$, per element of $X$ [16, Sec. 2]. Minimising the mutual information with respect to all channels that satisfy the threshold criterion achieves the task.111Note that the analogy between IBM and RDT is in spirit only. The technical difference is the distortion measure, which is a function of the output alphabet in the case of RDT while it is a function of the probability distribution of the outputs in the case of IBM.

The channel that achieves the rate in Eq. (1) can be found by the Lagrangian technique. The Lagrangian is defined as

[TABLE]

where $\beta$ is the Lagrange multiplier for the information constraint and $\lambda(x)$ are the Lagrange multipliers for the normalisation constraint of the conditional distribution $P(\tilde{X}|X)$. Taking the derivative of the Lagrangian with respect to the channel and setting it to [math] gives the expression for the channel as

[TABLE]

$D(.||.)$ in the exponent on the right hand side is the Kullback-Leibler divergence of two probability distributions $P(Y|x)$ and $P(Y|\tilde{x})$, and $Z$ is the normalising factor.

The setting of the quantum IBM is as follows. The input to the channel is the system $X$ which is described by a state $\rho_{XY}$ with side information $Y$. The channel acts on the $X$ part of this state, $\rho_{X}$. The output of the quantum channel is the system $\tilde{X}$ which is also an entangled state with the side information $Y$, $\tilde{\rho}_{\tilde{X}Y}$. Correlations in the state $\tilde{\rho}_{\tilde{X}Y}$ are measured by the quantum mutual information $I(\tilde{X};Y)_{\tilde{\rho}}:=S(\tilde{X})_{\tilde{\rho}}+S(Y)_{\tilde{\rho}}-S(\tilde{X}Y)_{\tilde{\rho}}$ where $S(\cdot)$ is the von Neumann entropy to base $e$. The Bottleneck constraint in the quantum case is $I(\tilde{X};Y)_{\tilde{\rho}}\geq J$, that is, a minimum amount of correlation $J$ between systems $\tilde{X}$ and $Y$ must be retained. This choice of measure for correlation between $\tilde{X}$ and $Y$ naturally generalizes the setup of the classical Bottleneck method; it includes both classical correlation and quantum entanglement between the two systems.

In the next section we provide the protocol for the quantum IBM which results in a conjecture for the quantum counterpart of the compression rate in Eq. (1). Section III contains the channel optimisation, resulting in the quantum counterpart of Eq. (3).

II A Protocol for the Quantum Information Bottleneck Method

In this section we explicitly describe a protocol that simulates the action of the channel in the setting discussed in the introduction by compression and decompression channels and classical communication with the assistance of entangled resources. We then define the compression rate for this protocol, before proving our main result.

Following the usual approach in RDT we call a rate-distortion pair $(r,J)$ achievable, where $r\geq 0$ is the rate and $J\geq 0$ is the distortion, if, roughly speaking, a lossy channel exists such that it can transmit a message, i.e., a state of the given input system, by sending $r$ bits with at the amount of distortion not exceeding $J$. The rate function $R(J)$ is then given as an infimum over achievable pairs. Of course, it is crucial to specify what exactly is meant by “lossy channel” and “achievable” in our context, and this is what we will discuss now.

Let us consider the entanglement-assisted protocol with noiseless classical communication illustrated in Fig. 1. The information of system $X$ to be compressed is represented by $n$ independently and identically distributed (i.i.d.) copies, $\rho_{X}^{\otimes n}$, of the density operator $\rho_{X}$, with a purification $\tau_{X^{\prime}X}$. The input, however, is correlated with a system $Y$ which contains our relevant information, and the state is denoted $\rho_{XY}$. The sender and receiver of the information share an entangled resource $\Phi_{T_{X}T_{\tilde{X}}}$, where the system $T_{X}$ is with the sender and the system $T_{\tilde{X}}$ is with the receiver. The sender acts on the input state, $\rho_{X}^{\otimes n}$, and the state of half of the entangled pair, $T_{X}$, with the compression map $\mathcal{E}_{n}:=\mathcal{E}_{X^{n}T_{X}\to W}$, where $W$ is a classical system of size at most $e^{nr}$, and $r$ is the communication rate. Then, the receiver acts on $W$ and the state of the other half of the entangled pair, $T_{\tilde{X}}$, with the decompression channel $\mathcal{D}_{n}:=\mathcal{D}_{WT_{\tilde{X}}\to{\tilde{X}}^{n}}$. The overall action $\mathcal{F}_{n}:=\mathcal{D}_{n}\circ\mathcal{E}_{n}$ of the compression-decompression channel, as specified above, defines an $(n,r)$ quantum rate-distortion code.

Given such $\mathcal{F}_{n}$, we consider the marginal operation, $1\leq i\leq n$,

[TABLE]

Then, for any $i$, we can define

[TABLE]

where $\mathcal{I}_{Y}$ is the identity channel acting on the system $Y$, and its partial traces are

[TABLE]

Using Eqs. (5) and (6) we can define the mutual information

[TABLE]

and its average over many uses of the channel,

[TABLE]

Now, for any $r,J\geq 0$, we call $(r,J)$ an achievable rate-distortion pair (for the Quantum Information Bottleneck Method) if there exists a sequence of $(n,r)$ quantum rate-distortion codes such that

[TABLE]

Finally, the rate function $R(J)$ is defined as

[TABLE]

Given these definitions, we can now express $R(J)$ in a more direct way in terms of one copy of the input state $\rho_{XY}$.

Conjecture 1**.**

Consider a memoryless quantum information source $\rho_{XY}$, where system $X$ is to be compressed such that after decompression it retains an amount of correlation $0\leq J\leq 1$ with system $Y$. The asymptotic Bottleneck rate function for entanglement-assisted lossy source coding with noiseless classical communication is given by

[TABLE]

under the conjecture that the expression on the right hand side of Eq. (11) is convex in $J$. Here, the quantum mutual information $I(X^{\prime};\tilde{X})$ is evaluated over the state

[TABLE]

such that $\tau_{X^{\prime}X}$ is the purification of the reduced state $\rho_{X}$ of the state $\rho_{XY}$, and

[TABLE]

.

While we are currently unable to remove the extra convexity assumption, we have verified it in numerical examples which are discussed in Appendix B.

Proof.

Here we follow the approach of [8, Theorem 2]. We temporarily denote the right-hand side of Eq. (11) as

[TABLE]

We need to show achievability of the rate , $R(J)\leq M(J)$, as well as optimality, $R(J)\geq M(J)$. We start with optimality. Let $(r,J)$ be an achievable rate-distortion pair and $\mathcal{F}_{n}$ a corresponding sequence of codes. We have for large $n$,

[TABLE]

The first inequality follows from the fact that the entropy of the uniform distribution, $nr$, is the upper bound of $S(W)$. The second inequality follows because entropy is nondecreasing under conditioning. The third inequality follows because the state of the system $WX^{\prime n}T_{\tilde{X}}$ is separable with respect to the classical system $W$ and therefore $S(W|X^{\prime n}T_{\tilde{X}})\geq 0$ [7, footnote 10]. The first equality follows from the definition of mutual information. The second equality follows since the state $\Phi_{T_{X}T_{\tilde{X}}}$ is in a tensor product with the state of the remaining input and therefore $I(X^{\prime n};T_{\tilde{X}})=0$. In the third equality we use the chain rule for mutual information. The fourth inequality follows from the data processing inequality. The fifth inequality follows from superadditivity of quantum mutual information [8, Lemma 15]. The sixth inequality follows from the definition of $M(J)$, where we use the channel $\mathcal{N}_{X\rightarrow\tilde{X}}=\mathcal{F}_{n}^{(i)}$. In the seventh inequality we have used conjecture of convexity of $M$. Finally the last inequality follows for large $n$ from Eq. (9), using that $M(J)$ is a nondecreasing function of $J$. The rate function is nondecreasing in $J$, because for any $J^{\prime}>J$ the domain of minimisation in Eq. (11) becomes smaller, which implies that the rate function can only become larger. Thus, since $(r,J)$ was arbitrary, (15) implies $R(J)\geq M(J)$.

Achievability follows from an application of the Quantum Reverse Shannon Theorem (QRST) [4, 5, 10]. The general form of the QRST states that a quantum channel can be simulated by an unlimited amount of shared entanglement and an amount of classical communication equal to the channel’s entanglement-assisted classical capacity. Specifically, in our case, fix $J>0$ and let $\mathcal{N}_{X\rightarrow\tilde{X}}$ be the optimum channel at which the minimum in Eq. (14) is attained. For a given $\epsilon>0$, we use the QRST to construct a sequence of channels $\mathcal{F}_{n}=\mathcal{D}_{n}\circ\mathcal{E}_{n}$ such that they are close to an $n$-fold application of $\mathcal{N}_{X\rightarrow\tilde{X}}$, in the sense that

[TABLE]

where $\tau_{X^{\prime}\tilde{X}}$ is defined in Eq. (12) and

[TABLE]

According to the QRST we can construct such a sequence of channels, $\mathcal{F}_{n}$, using classical communication at rate $r=I(X^{\prime};\tilde{X})_{\tilde{\tau}}$, given an unlimited amount of entanglement for a known tensor power input, which is our setting, c.f. [4, Fig.2, top right]222Notice that the QRST in Fig. 2 of [4] was done in two ways, a so-called feedback and a non-feedback simulation, respectively. The former means that the environment of the simulated channel is in the possession of the sender, while the latter means some part of the environment of the channel will end up on the receiver’s side. In both cases, if unlimited entanglement is available, the rate is given by the single-letter formula, the above mutual information. In the case of the non-feedback simulation of the channel $\mathcal{N}_{X\to\tilde{X}}$, the feedback simulation protocol [4, Theorem 3(a)] is used, namely the isometry $\mathcal{U}_{\mathcal{N}}^{X\to E\tilde{X}}$ composed with the isometry $V^{E^{n}\to E_{A}E_{B}}$, is simulated by QRST, such that systems $E_{A}$ and $X$ are with the sender and the systems $E_{B}$ and $\tilde{X}$ are with the receiver. Tracing out the environment from this channel gives the desired channel, $\mathcal{N}_{X\to\tilde{X}}$. The result listed in Fig. 2 of [4] can be obtained from Theorem 3(b) there as follows. In the presence of an unlimited shared entanglement, condition (21) in [4] trivially holds and the maximum in Eq. (23) there is always attained by the first of the two terms. To obtain the desired expression of the mutual information from (20) and (23) there, it is enough to choose $E^{n},E_{A}$ and $E_{B}$ such that the isometry $V$ acts trivially and we have $I(R^{n}:B^{n}E_{B})=I(R^{n}:B^{n})$. Finally, the single letter formula $I(R:B)$ that we use follows from the fact that we have a known tensor-power input. 333Notice that the rates shown in Fig. 2 of [4] are quantum communication rates which is why they are different by a factor of 1/2 from what we use. The extra factor drops because 1-qubit channels can be simulated with 2 bit classical channels in the presence of entanglement using the teleportation protocol.. From Eq. (16) and the fact that $I(\tilde{X};Y)\geq J$ for the channel $\mathcal{N}_{X\rightarrow\tilde{X}}$, one can show (see Lemma 1 in Appendix A) that Eq. (9) is fulfilled with $J-\delta$ instead of $J$, where $\delta\rightarrow 0$ as $\epsilon\rightarrow 0$. Hence, $(r,J-\delta)$ is achievable and $R(J-\delta)\leq M(J)$.

We have now shown that

[TABLE]

From this it follows that

[TABLE]

Since $R$ is nondecreasing, by Eqs. (9) and (10), and $M$ is continuous (a property that follows from the conjecture of convexity), $M(J)=R(J)$ for all $J$.

∎

Remark 1: In Eq. (11) the Quantum Reverse Shannon Theorem is used, which in addition uses shared entanglement (“entanglement assistance”) to generate a protocol that achieves the rate. However, the requirements for the Quantum Reverse Shannon Theorem are much more stringent than those of the Bottleneck method. Therefore, it might be possible to find a rate function without entanglement assistance. This is still an open problem.

Remark 2: The systems $XY$ and $XX^{\prime}$ are understood to be in separate setups; the states $\tilde{\rho}_{\tilde{X}Y}$ and $\tilde{\tau}_{X^{\prime}\tilde{X}}$ are not partial traces of a common parent state, but they involve an identical channel $\mathcal{N}$ (“two separate experiments using the same apparatus”). The QRST applies only to the system $XX^{\prime}$. To prove Eq. (11), these two setups need to be connected, which is done in Lemma 1 in Appendix A.

Remark 3: Generally, rate distortion functions obtained from the definition in Eq. (10) are non-increasing functions of the distortion, whereas the RHSs of Eqs. (11) and (1) are both non-decreasing functions. This is not a fundamental difference between RDT and the Bottleneck method. It is merely due to the fact that in the IBM the constraint of minimisation is chosen to be the amount of correlation preserved, $I(\tilde{X};Y)\geq J$, while in RDT the constraint of minimisation is the average loss of information, $\langle d(X;\tilde{X})\rangle\leq D$, for some fixed $D$ and some distortion measure $d(X;\tilde{X})$, characterising the noise introduced by a channel $\mathcal{N}_{X\to\tilde{X}}$ to a state $\rho_{X}$.

The formulation of the IBM can easily be changed to using a constraint on the loss of correlation such as $I(X;Y)-I(\tilde{X};Y)\leq D$, in which case the rate function is a non-increasing function of $D$. Rather than changing it, the structure of the minimisation constraint is kept in line with the classical IBM.

Remark 4: As discussed above, in a lossy compression-decompression protocol the minimisation is performed over all channels satisfying a certain criterion, see Eq. (11). Although the functions $I(\tilde{X};X^{\prime})$ and $I(\tilde{X};Y)$ are convex in the channel, the optimisation problem in Eq. (11) is not of convex type due to the sign of the inequality in the constraint ($\geq$ rather than $\leq$). Just as in the classical case, it is a so-called “reverse convex problem” (see, e.g., [15]), for which many of the standard results, such as strong duality or convexity of the resulting function in $J$, are not known to hold or apply only in a weaker form. Nevertheless, we present numerical evidence for convexity of the RHS in Appendix B. We also show that the minimum in Eq. (11) is actually attained at the boundary $I(\tilde{X};Y)_{\tilde{\rho}}=J$ (see Lemma 2 in Appendix C), which makes it useful to search for local optima with the method of Lagrange multipliers, as we show in the next section.

III The Optimisation Problem for the Quantum Information Bottleneck Method

We now proceed to the optimisation problem. In order to formulate the Lagrangian corresponding to the quantum counterpart of Eq. (2), we need to choose a suitable way of representing the channel $\mathcal{N}$, e.g., the Kraus operators or the Choi-Jamiołkowski representation. It turns out that indeed the most compact and convenient way to compute the derivatives of the Lagrangian is with respect to the Choi-Jamiołkowski representation defined as

[TABLE]

where $\tilde{\Phi}_{X^{\prime}X}:=\sum_{i,j=0}^{d-1}|i\rangle\langle j|_{X^{\prime}}\otimes|i\rangle\langle j|_{X}$. For the rate function given in Eq. (11) one can write the Lagrangian

[TABLE]

where $t_{X}$ denotes the partial transpose with respect to the basis $\{\ket{i}_{X}\}_{i}$ on $X$; here $\beta$ is the Lagrange multiplier for the constraint of minimisation and the Hermitian operator $\Lambda_{X}$ is the Lagrange multiplier to guarantee that the channel is a completely positive trace preserving map. The states in Eq. (21) can be written as functions of the Choi-Jamiołkowski state of the channel $\Psi_{X\tilde{X}}$. The joint state $\tau_{X^{\prime}X}$ in Eq. (21) can be written as

[TABLE]

where $\rho^{X}_{X^{\prime}}$ is the same state as $\rho_{X}$ acting on the Hilbert space $\mathcal{H}_{X^{\prime}}$ of the system $X^{\prime}$.

By similar considerations, one can show that the joint state

[TABLE]

can be written as

[TABLE]

where we have chosen a maximally entangled state $\Phi_{Y^{\prime}Y}$ and a corresponding transpose $t_{Y^{\prime}}$. In the third term of Eq. (21) the dependence on the channel state is already explicit.

Let

[TABLE]

and

[TABLE]

Taking the derivative of the Lagrangian in Eq. (21) with respect to the channel and setting it to zero (for details, see Appendix C) gives the optimum channel as

[TABLE]

Note that this determines $\Psi_{X\tilde{X}}^{t_{X}}$ implicitly since it also appears on the right hand side of this equation in $\tilde{\rho}_{\tilde{X}}$ and in the definition of $D_{X\tilde{X}}^{\beta Y}$ (for details, see Appendix C). In order to find the optimum channel, Eq. (26) needs to be solved iteratively for $\Psi_{X\tilde{X}}^{t_{X}}$. The solution also depends on the unknown Lagrange multipliers $\Lambda_{X},\beta$ associated with the constraints of the problem; these need to be determined in a further optimisation step. We comment on a possible algorithm to that end in Appendix C.

Eq. (26) reduces to its classical counterpart in Eq. (3) in the case of diagonal density operators. To see this, consider the diagonal case where the density operators reduce to probability distributions. From Eq. (26) it follows that

[TABLE]

with $\lambda(x)$ being the same normalisation Lagrange multiplier as in Eq. (2). Notice that since $H(Y|X=x)=-\sum_{y}P(y|x)\log P(y|x)$ depends only on $x$ but not on $\tilde{x}$, it can be absorbed into $\lambda(x)$. Defining

[TABLE]

Eq. (27) becomes

[TABLE]

which is the same classical channel as Eq. (3), with all the extra terms being absorbed into the normalisation factor. This also shows that $D_{X\tilde{X}}^{\beta y}$ is a quantum operator corresponding to the distance measure in the classical Bottleneck method. The idea of distance operators has been used in a number of quantum information processing tasks [17, 9], however the $D_{X\tilde{X}}^{\beta Y}$ is particular to the present setting. Eq. (26) can be used in principle to compute numerical values of quantum channels using iterative algorithms, akin to their classical counterparts by methods introduced by Blahut and Arimoto [6, 2].

Finally, we would like to remark that a previous result in the domain of quantum information theory was inspired by the classical Information Bottleneck Method [12], and indeed some of the technical steps in Section III of our paper are very similar. Here we clarify the difference between our contribution and that of [12]. To reflect on our contribution, consider that in classical communication, Shannon’s channel capacity theorem gives a single-letter formula that fully characterises a communication channel. This allows one to interpret the mutual information between the input and the output of a channel as a communication rate, as was done in [16]. In quantum communication theory the situation is different since quantum channels (except for very few cases) are typically not characterised by such single-letter quantities. Therefore, in order to give a communication theoretic interpretation to an entropic quantity such as the one that we (and [12]) use, one has to prove that the quantity corresponds to a rate in a communication scenario by giving a coding scenario to which the rate refers. We have done this, up to the previously stated conjecture, by designing a setup that is compatible with the Quantum Reverse Shannon Theorem (notice, e.g., the unlimited amount of entanglement), and by proving the lemma given in Appendix A. This lemma shows that in our setting the Quantum Reverse Shannon Theorem can be invoked. In Ref. [12], the focus there was on applying a quantum extension of the Information Bottleneck Method to predictive filtering, whereas we focus here on the communication scenario.

IV Conclusion and Outlook

This paper introduced the quantum extension of the Information Bottleneck Method. This method compresses the data such that only the information relevant with respect to some given variable is preserved. We derive a lower bound to the compression rate of relevant quantum information. The problem was formulated as a communication channel problem and the rate was shown to be achievable by explicitly constructing a channel achieving it. Just like in the classical case, the compression rate of the quantum Information Bottleneck Method is lower than that given by quantum rate distortion theory. Several conceptual issues arose from the structural differences between the mathematical formalism of quantum theory and classical probability theory which were discussed and solutions were presented.

Some open questions remain. Our proof of Eq. (11) relied on an unproven conjecture (convexity of the expression on the right hand side of Eq. (11) in $J$). While this seems to be fulfilled in examples (cf. Appendix B), a proof of this property is currently missing.

In Appendix B a simple algorithm is used to compute the optimum channel and thus the rate function $R(J)$ in low dimensional systems; but for systems of realistic size a more efficient algorithm would be required. This might be based on numerically solving the implicit equation (26).

Acknowledgment

The authors are grateful to Henning Bostelmann for discussions on the subject and for assistance with the numerics in Appendix B. Further, D. C. would like to thank Roger Colbeck and Robert Koenig for helpful discussions. P. K. would like to thank David Sutter and Renato Renner for comments on earlier versions of the manuscript.

S. S. is supported by the National Science Foundation of China through Grant No. 11675136, the Hong Kong Research Grant Council through Grant No. 17326616, the EPSRC National Quantum Technology Hub in Networked Quantum Information Technologies. This publication was made possible through the support of a grant from the John Templeton Foundation. The opinions expressed in this publication are those of the authors and do not necessarily reflect the views of the John Templeton Foundation. P. K. acknowledges funding from the Swiss National Science Foundation (through the National Centre of Competence in Research “Quantum Science and Technology” and through Project No. 200020_165843), from the European Research Council (grant 258932) and through COST MP1209 ”Thermodynamics in the quantum regime”.

K.W. acknowledges support through EPSRC grant I013717/1.

Appendix A Lemmas for the proof of the Bottleneck rate function

The following lemma is relevant for the proof of achievability of the communication rate. It has the same application as in Theorem 19 of [19] and Lemma 1 of [8], but has been adapted to the distortion criterion for the quantum Bottleneck method.

Lemma 1**.**

Let $\eta(\lambda):=-\lambda\log\lambda$. There exists a constant $k>0$ depending only on the dimension of $\mathcal{H}_{\tilde{X}Y}$ such that the following holds:

Let $0<J\leq I(X;Y)_{\rho}$ be fixed. Let a quantum channel $\mathcal{N}_{X\to\tilde{X}}$ be such that if we apply the channel to the system $X$ and an identity channel $\mathcal{I}_{Y}$ on the system $Y$ the effect will meet the condition $I(\tilde{X};Y)_{\tilde{\rho}}\geq J$, where $\tilde{\rho}_{\tilde{X}Y}$ is given by (23). Further, let $\mathcal{F}_{n}$ be a sequence of quantum channels from the space of density matrices $\mathfrak{D}(\mathcal{H}_{X^{\prime}X}^{\otimes n})$ to $\mathfrak{D}(\mathcal{H}^{\otimes n}_{X^{\prime}\tilde{X}})$ such that

[TABLE]

for some $0\leq\epsilon<\frac{1}{e}$ and large enough $n$, where $\tilde{\tau}_{X^{\prime}\tilde{X}}$ and $\tilde{\sigma}_{X^{\prime n}\tilde{X}^{n}}$ are given by (12) and (17), respectively.

Then, for large enough $n$ and $\delta:=k\eta(\epsilon)$, we have

[TABLE]

Proof.

Adding and subtracting $I(\tilde{X};Y)_{\tilde{\rho}}$ to the left hand side of Eq. (31) and using the triangle inequality, we obtain

[TABLE]

Since we assumed that $I(\tilde{X};Y)_{\tilde{\rho}}\geq J$, what we need to show is that

[TABLE]

with some $k>0$ and $\eta$ as above. Inserting $\bar{I}_{n}$ given in Eq. (8), we have

[TABLE]

Hence, it suffices to show that for all $1\leq i\leq n$,

[TABLE]

In particular, we will prove bounds of the above type on $|S(\tilde{\rho}_{\tilde{X}})-S(\tilde{\sigma}_{\tilde{X}_{i}})\lvert$ , $|S(\tilde{\rho}_{\tilde{X}Y})-S(\tilde{\sigma}_{\tilde{X}_{i}Y_{i}})\lvert$ and $|S(\rho_{Y})-S(\tilde{\sigma}_{Y_{i}})|$ for all $i$. We start with the first of these. To prove this inequality, we recall that $\tau_{X^{\prime}X}$ is a purification of $\rho_{x}$. Now let $\rho_{X^{\prime}Y^{\prime}XY}$ be a purification of $\rho_{XY}$, then there is a Hilbert space $\mathcal{\hat{H}}$ and a unitary $U:\mathcal{H}_{X^{\prime}}\oplus\mathcal{\hat{H}}\rightarrow\mathcal{H}_{X^{\prime}Y^{\prime}Y}$ such that

[TABLE]

where $\tau_{X^{\prime}X}$ is extended to the orthogonal complement of $\mathcal{H}_{X^{\prime}}$ by zeros. Then, (30) implies

[TABLE]

where $\tilde{\rho}_{X^{\prime}Y^{\prime}\tilde{X}Y}=(\mathcal{N}_{X\rightarrow\tilde{X}}\otimes\mathcal{I}_{X^{\prime}Y^{\prime}Y})(\rho_{X^{\prime}Y^{\prime}XY})$. Further, one computes that

[TABLE]

[TABLE]

To summarize, we found that

[TABLE]

Using monotonicity of the trace norm under partial trace, we find that

[TABLE]

Moreover,

[TABLE]

again using the monotonicity of the trace norm under partial trace. This implies that $\|\tilde{\rho}_{\tilde{X}}-\tilde{\sigma}_{\tilde{X}_{i}}\|_{1}\leq\epsilon$.

Now, by the Fannes Inequality the following bound holds for all $1\leq i\leq n$:

[TABLE]

where $k^{\prime}$ is the dimension of $\mathcal{H}_{\tilde{X}}$. Then, using the bound $\|\tilde{\rho}_{\tilde{X}}-\tilde{\sigma}_{\tilde{X}_{i}}\|_{1}\leq\epsilon$, we find for all $1\leq i\leq n$,

[TABLE]

where the last inequality uses the fact that $\eta(\epsilon)\geq\epsilon$ for $0\leq\epsilon<\frac{1}{e}$, and where $\hat{k}^{\prime}$ is defined in terms of the constants in the first inequality, including $k^{\prime}$.

With a similar method, one can prove that for all $1\leq i\leq n$,

[TABLE]

where $\hat{k}^{\prime\prime},\hat{k}^{\prime\prime\prime}$ also depend on the dimensions of $\mathcal{H}_{\tilde{X}Y}$ and $\mathcal{H}_{Y}$, respectively.

Combining Eq. (41), (42) and (43) we find for all $1\leq i\leq n$ that Eq. (A) holds for a constant $k$ that includes the constants in the three estimates above and depends only on the dimension of $\mathcal{H}_{\tilde{X}Y}$.

Hence, we obtain for large enough $n$ that $\bar{I}_{n}(\tilde{X};Y)_{\tilde{\sigma}}\geq J-k\eta(\epsilon)$. ∎

Appendix B Numerical examples

The aim of this appendix is to compute the communication rate as a function of $J$ for some examples, using a numerical optimisation algorithm for evaluating the right-hand side of Eq. (11). In particular, in all these examples the rate function turns out to be convex in $J$. Consider the following normalized version of the rate

[TABLE]

where now $0<J<1$. In the following examples the systems $X$ and $Y$ are described by two-dimensional Hilbert spaces spanned by the basis $\lvert\uparrow\rangle,\lvert\downarrow\rangle$.

To find the optimum numerically, a simple random search algorithm is used [1]. It initially chooses a number of channels at random (in terms of their Kraus operators) and computes the related mutual information, then randomly varies those channels with the lowest $I(X^{\prime},\tilde{X})$ further until a stable optimum is reached.

This algorithm is applied to three classes of input states $\rho_{XY}$: The first example is a “classical” state, i.e., a state without entanglement between the systems $X$ and $Y$, given by the density matrix

[TABLE]

where $p_{1},p_{2},p_{3},p_{4}$ are nonnegative numbers with $p_{1}+p_{2}+p_{3}+p_{4}=1$. The second example is a state with entanglement between $X$ and $Y$, namely,

[TABLE]

Finally, the third example is again a state with entanglement defined as

[TABLE]

with the normalized vectors

[TABLE]

and nonnegative numbers $p_{1},p_{2}$ with $p_{1}+p_{2}=1$. The plots presented in Figs. 2–5 show the rate as a function of distortion. The blue lines correspond to the curves $\hat{R}(J)=J$ and $\hat{R}(J)=\frac{1}{2}J$ and are introduced for comparison with the actual result (red line).

The four plots show that the function $\hat{R}(J)$, given by (44), is indeed a convex function in $J$ for the specific choices of initial state $\rho_{XY}$ and within the limits of the numerical approximation. Note that in Figs. 3–5, the graph does not appear to be differentiable at the point $\hat{R}=\frac{1}{2}$. This seems to be a common feature of the examples $\rho_{XY}^{(2)},\rho_{XY}^{(3)}$, but we do not currently have an analytic explanation for this behaviour. Note that in Fig. 2, one has $M(1)=\frac{1}{2}$, while in Figs. 3–5, one has $M(1)=1$. In other words, in the case of a “classical” (non entangled) state $\rho_{XY}$, there is a channel such that $I(\tilde{X};Y)=I(X;Y)$ and $I(\tilde{X};X^{\prime})=\frac{1}{2}I(X;X^{\prime})$. To obtain an analytic expression of this channel we proceed as follows.

The initial state is $\rho_{XY}^{(1)}$. Then, $\rho_{X}^{(1)}=\operatorname{Tr}_{Y}\rho_{XY}^{(1)}=(p_{1}+p_{3})\lvert\uparrow\rangle\langle\uparrow\rvert+(p_{2}+p_{4})\lvert\downarrow\rangle\langle\downarrow\rvert$ and a purification is given by $\tau_{X^{\prime}X}^{(1)}=\lvert w\rangle\langle w\rvert$ with $\lvert w\rangle=\sqrt{p_{1}+p_{3}}\lvert\uparrow\uparrow\rangle+\sqrt{p_{2}+p_{4}}\lvert\downarrow\downarrow\rangle$. Our ansatz for the channel is

[TABLE]

with $K_{1}:=\lvert\uparrow\rangle\langle\uparrow\rvert-\lvert\downarrow\rangle\langle\downarrow\rvert$ and $K_{2}:=\lvert\uparrow\rangle\langle\uparrow\rvert+\lvert\downarrow\rangle\langle\downarrow\rvert$. Applying this channel to $\rho_{XY}^{(1)}$ and $\tau_{X^{\prime}X}^{(1)}$, we obtain $\tilde{\rho}_{\tilde{X}Y}^{(1)}=\rho_{XY}^{(1)}$ and

[TABLE]

Note that here $\tilde{\tau}_{\tilde{X}}^{(1)}=\tau_{X^{\prime}}^{(1)}=\tau_{X}^{(1)}=\rho_{X}^{(1)}$ are all matrices, but they act on different Hilbert spaces. With this choice of channel and initial state it is clear that $I(\tilde{X};Y)_{\tilde{\rho}^{(1)}}=I(X;Y)_{\rho^{(1)}}$. In order to show that $I(\tilde{X};X^{\prime})_{\tilde{\tau}^{(1)}}=\frac{1}{2}I(X;X^{\prime})_{\tau^{(1)}}$ we compute the von Neumann entropies in $I(\tilde{X};X^{\prime})=S(\tilde{X})+S(X^{\prime})-S(\tilde{X}X^{\prime})$ and in $I(X;X^{\prime})=S(X)+S(X^{\prime})-S(XX^{\prime})$ using the fact that $S(XX^{\prime})=0$, since $\tau_{X^{\prime}X}$ is pure, and that $\rho_{X}^{(1)},\tilde{\tau}_{X^{\prime}\tilde{X}}^{(1)}$ are diagonal. After evaluating the matrix functions, we obtain

[TABLE]

as desired. Therefore, $\hat{R}(1)\leq 1/2$ for this class of states, consistent with the graph.

This somewhat unexpected feature may be understood as follows: In order to transmit the $X$ part of the non-entangled state $\rho_{XY}^{(1)}$ perfectly, a classical channel of 1 bit capacity is sufficient. By the usual quantum teleportation result, this corresponds to a quantum channel of only $\frac{1}{2}$ qbit capacity, if shared entanglement is available in abundance.

While the simplistic random search algorithm that we used in this appendix is sufficient for demonstrating the essential quantum features of the system, it would be too inefficient to compute the rate function $R(J)$ for systems with larger Hilbert space dimension. More sophisticated numerical schemes would be required to that end, e.g., taking the gradient of the objective function into account. Appendix C describes a Lagrangian method that may be used to this end.

Appendix C The Optimal Map

In this appendix we describe a method for finding the optimum channel in Eq. (11). We first remark that the optimum is actually attained at the boundary of the constraint region, $I(\tilde{X};Y)=J$. This is heuristically clear, as for a channel in the interior ($I(\tilde{X};Y)>J$) one could always reduce the channel’s communication rate by a small amount at the expense of a slightly decreased quality measure $I(\tilde{X};Y)$, staying inside the region of the constraint. We give a more formal argument to this end.

Lemma 2**.**

The minimum in Eq. (11) is attained on the hypersurface $I(\tilde{X};Y)=J$.

Proof.

Suppose that the minimum was attained at a channel $\mathcal{N}_{1}$ such that $I(\tilde{X};Y)_{\mathcal{N}_{1}}>J$ (here the subscript denotes that the state $\tilde{\rho}_{\tilde{X}Y}$ is computed with respect to the channel $\mathcal{N}_{1}$). Let $\mathcal{N}_{0}$ be the channel $\mathcal{N}_{0}(\sigma):=(\operatorname{dim}\mathcal{H}_{\tilde{X}})^{-1}\operatorname{Tr}(\sigma)\cdot\boldsymbol{1}$; we note that $I(\tilde{X};Y)_{\mathcal{N}_{0}}=0=I(\tilde{X};X^{\prime})_{\mathcal{N}_{0}}$. For $0<\lambda<1$, let us consider a new channel $\mathcal{N}_{\lambda}:=\lambda\mathcal{N}_{1}+(1-\lambda)\mathcal{N}_{0}$. From convexity of the mutual information in the channel, it follows that

[TABLE]

On the other hand, from continuity considerations we know that $I(\tilde{X};Y)_{\mathcal{N}_{\lambda}}>J$ for $\lambda$ close enough to 1, so that also $\mathcal{N_{\lambda}}$ fulfills the constraint in this region. Hence $M(J)<I(\tilde{X};X^{\prime})_{\mathcal{N}_{1}}$, and $\mathcal{N}_{1}$ is not the position of the minimum. ∎

Therefore, we deal with an optimisation problem of a function over a constraint hypersurface, which makes it useful to look for local extrema with the method of Lagrange multipliers. We use the Lagrangian in Eq. (21),

[TABLE]

and look for zeros of its derivatives. We use the Choi-Jamiołkowski representation

[TABLE]

of the channel in order to compute the derivative of the Lagrangian, where $\Phi_{X^{\prime}X}:=\sum_{i,j=0}^{d-1}|i\rangle\langle j|_{X^{\prime}}\otimes|i\rangle\langle j|_{X}$ is the Choi-Jamiołkowski matrix corresponding to the identity channel from the Hilbert space $\mathcal{H}_{X^{\prime}}$ to $\mathcal{H}_{X}$, and $\mathcal{N}_{X\to{\tilde{X}}}$ is the channel that simulates the compression-decompression process. $\Lambda_{X}$, an operator on the Hilbert space $\mathcal{H}_{X}$, is the Lagrange multiplier introduced for the normalisation of the channel $\Psi_{X\tilde{X}}^{t_{X}}$. Considering the definition of the mutual information, to compute the derivative $\frac{\delta\mathcal{L}}{\delta\Psi_{X\tilde{X}}^{t_{X}}}$ of the Lagrangian, we need to compute the following derivatives,

[TABLE]

Notice that the functions in the numerator of the expressions in the first two equations in (49) are independent of the channel and, hence, the derivatives are zero. For the five remaining ones we note that for a Hermitian operator, $A$, and a function, $f$, which is analytic on the spectrum of $A$, the directional derivative of $\operatorname{Tr}[f(A)]$ is given by

[TABLE]

with the direction given by the operator $B$ and $f^{\prime}$ being the first derivative of the function $f$. (This follows from analytic functional calculus, expanding $\operatorname{Tr}f(A+\epsilon B)$ in a Taylor series around $\epsilon=0$.) Specifically, let us define $f(z):=z\log(z)$. Since the derivative of our function $f$ is given by $f^{\prime}(\cdot)=(1+\log)(\cdot)$, using (50) and (22), we have

[TABLE]

where

[TABLE]

Likewise, we can compute

[TABLE]

In order to compute the derivative for $S(\tilde{X})_{\tilde{\rho}}$ and $S(\tilde{X}Y)_{\tilde{\rho}}$ we use Eqs. (III) and (50), and we find

[TABLE]

and

[TABLE]

where

[TABLE]

For the last derivative we have

[TABLE]

Putting all the terms together we have

[TABLE]

Setting this expression to zero ($\frac{\delta\mathcal{L}}{\delta\Psi_{x{\tilde{X}}}^{t_{X}}}[B_{X\tilde{X}}]=0$), we find

[TABLE]

Rearranging left and right hand sides of this equation, we find

[TABLE]

This holds for all directions $B_{X\tilde{X}}$, which implies

[TABLE]

By performing the partial trace on the left hand side of this expression, we obtain

[TABLE]

Simplifying this expression further, we find

[TABLE]

Let us denote

[TABLE]

and the normalisation term $\tilde{\Lambda}_{X}:=\rho_{X}^{-1/2}\Lambda_{X}\rho_{X}^{-1/2}$. Exponentiating both sides of Eq. (61), we obtain

[TABLE]

Using Eq. (22), we arrive at the expression for the Choi-Jamiołlkowski matrix corresponding to the channel,

[TABLE]

Note that this is an implicit equation in $\Psi_{X\tilde{X}}$ since it also appears on the right hand side of this expression. To find the optimum channel, Eq. (64) needs to be solved iteratively for $\Psi_{X\tilde{X}}$. Note that the unknown Lagrange multiplier $\Lambda_{X}$, which is associated with the normalisation constraint, is still contained in this equation. An algorithm that recursively computes the channel might work as follows. Starting with a guess for the channel $\Psi_{X\tilde{X}}$ and normalising it, this guess is inserted into Eq. (60) to compute a self-consistent value for $\Lambda_{X}$. This would allow to compute $\tilde{\rho}_{\tilde{X}}$ and $\tilde{\rho}_{\tilde{X}Y}$ from the channel, and hence give an approximation for all quantities that enter the right hand side of Eq. (26). Thus, a new approximation for the left-hand side is obtained, i.e., the channel $\Psi_{X\tilde{X}}$. When this procedure is repeated iteratively, an optimal channel $\Psi_{X\tilde{X}}$ is obtained at a given value for $\beta$. Repeating this procedure for different values of $\beta$ and optimising under the constraint $I(\tilde{X};Y)_{\tilde{\rho}}\geq J$ yields the minimum in (11).