A de Bruijn identity for discrete random variables

TL;DR

This paper introduces a new operation called beamsplitter addition for discrete non-negative integer variables, providing a classical proof of a heat equation and de Bruijn identity related to geometric distributions.

Contribution

It defines a novel scaled convolution operation for discrete variables and derives a classical proof of related heat and de Bruijn identities.

Findings

Beamsplitter addition acts as a scaled convolution for discrete variables.

Provides a classical proof of the heat equation for geometric distributions.

Establishes a de Bruijn identity for discrete random variables.

Abstract

We discuss properties of the "beamsplitter addition" operation, which provides a non-standard scaled convolution of random variables supported on the non-negative integers. We give a simple expression for the action of beamsplitter addition using generating functions. We use this to give a self-contained and purely classical proof of a heat equation and de Bruijn identity, satisfied when one of the variables is geometric.

Equations

$$p_{{\mathbf{X}}_c}(\mathbf{r})$$

$$p_X[n]$$

$$X{⊞}_{\eta }Y={\mathcal{T}}^{-1}\left(\sqrt{\eta }\mathcal{T}(X)+\sqrt{1-\eta }\mathcal{T}(Y)\right),$$

$$H_X(t):=\sum _{m=0}^{\infty }{t}^m\left(\frac{\mathbb{E}{\left(X\right)}_{(m)}}{m!}\right),$$

$${\tilde{H}}_X(t):=\sum _{m=0}^{\infty }\frac{t^m}{m!}\left(\frac{\mathbb{E}{\left(X\right)}_{(m)}}{m!}\right).$$

$${\varphi }_{{\mathbf{X}}_c}(t)$$

$${\tilde{\varphi }}_{{\mathbf{X}}_c}(t):=\sum _{m=0}^{\infty }\frac{t^m}{m!}\left(\frac{\mathbb{E}|{\mathbf{X}}_c{|}^{2m}}{m!}\right).$$

$${\tilde{H}}_X(t)=\exp (-t){\tilde{\varphi }}_{{\mathbf{X}}_c}(t).$$

$$H_X(t)=\frac{1}{1+t}{\varphi }_{{\mathbf{X}}_c}\left(\frac{t}{1+t}\right),$$

$${\lambda }_X=H_X^{\prime }(0)={\varphi }_{{\mathbf{X}}_c}^{\prime }(0)-{\varphi }_{{\mathbf{X}}_c}(0)=\mathbb{E}|{\mathbf{X}}_c{|}^2-1.$$

$${\tilde{H}}_Z(t)={\tilde{H}}_X(\eta t){\tilde{H}}_Y((1-\eta )t).$$

$$\frac{1}{s-1}{H}_Z\left(\frac{1}{s-1}\right)=\left[L\left(M_X^{(\eta )}\times M_Y^{(1-\eta )}\right)\right]\left(s\right),$$

$$\left[L{M}_{{\mathbf{X}}_c}^{(\eta )}\right](s)$$

$$\left[L{M}_{{\mathbf{Y}}_c}^{(1-\eta )}\right](s)$$

$${\lambda }_Z=\eta {\lambda }_X+(1-\eta ){\lambda }_Y.$$

$$\exp (\eta {\lambda }_{X}t)\exp ((1-\eta ){\lambda }_{Y}t)=\exp ({\lambda }_{Z}t),$$

$$\frac{1}{s-1}{H}_Z\left(\frac{1}{s-1}\right)=\frac{1}{s-(1+{\lambda }_Z)},$$

$$\frac{\partial }{\partial \eta }{p}_{Z_{\eta }}[n]:=\Delta \left(\frac{n}{\eta }\left(p_{Z_{\eta }}[n-1]{\lambda }_Y-p_{Z_{\eta }}[n](1+{\lambda }_Y)\right)\right),$$

$$p_X^{+}[n]=\frac{(n+1)p_X[n+1]}{{\lambda }_X}\text{mbox}and{p}_X^{-}[n]=\frac{(n+1)p_X[n]}{1+{\lambda }_X}.$$

$$\frac{\partial }{\partial \eta }D(Z_{\eta }\Vert G_{\eta })$$

$$=\frac{{\lambda }_Y(1+\lambda (\eta ))}{\eta }D(p_{Z_{\eta }}^{-}\Vert p_{Z_{\eta }}^{+})+\frac{(1+{\lambda }_Y)\lambda (\eta )}{\eta }D(p_{Z_{\eta }}^{+}\Vert p_{Z_{\eta }}^{-}),$$

$$D(X\Vert G)=H(G)-H(X).$$

$$D(X\Vert G)\le {\lambda }_X(1+{\lambda }_X)\left(D(p_X^{-}\Vert p_X^{+})+D(p_X^{+}\Vert p_X^{-})\right).$$

$$D(Z_{\eta }\Vert G)\le \eta D(X\Vert G)+(1-\eta )D(Y\Vert G)=\eta D(X\Vert G),$$

$$\frac{D(Z_{\eta }\Vert G)-D(X\Vert G)}{\eta -1}\ge D(X\Vert G).$$

$${\rho }_X[n]$$

$$J^{+}(X)$$

$$J^{-}(X)$$

$$\frac{\partial }{\partial \eta }{p}_{Z_{\eta }}[n]:=\frac{1+{\lambda }_X}{\eta }\Delta \left(p_{Z_{\eta }}[n]{\rho }_{Z_{\eta }}[n]\right).$$

$$D(X\Vert G)\le (1+{\lambda }_X)\left(J^{+}(X)+J^{-}(X)\right).$$

$$D({\mathbf{X}}_c\Vert {\mathbf{Y}}_c)\le D(X\Vert Y).$$

$$p_{{\mathbf{X}}_c}(\mathbf{r})\log \left(\frac{p_{{\mathbf{X}}_c}(\mathbf{r})}{p_{{\mathbf{Y}}_c}(\mathbf{r})}\right)$$

Full text

I Introduction and notation

Stam [1] showed that addition of independent continuous random variables satisfies the de Bruijn identity [1, Eq. (2.12)], in that the derivative of entropy under the addition of a normal is Fisher Information. This identity lies at the heart of many analyses of entropy under addition, including Stam’s proof of Shannon’s Entropy Power Inequality (EPI) [2] and Barron’s information theoretic Central Limit Theorem [3], and Madiman and Barron’s proof of monotonicity of entropy under addition of independent identically-distributed (i.i.d.) continuous random variables [4]. These results have the Gaussian distribution at their heart, relating to the Gaussian maximum entropy property and closure of the Gaussian family under addition (“Gaussian $+$ Gaussian $=$ Gaussian”). The de Bruijn identity follows because the densities in question satisfy the heat equation [1, Eq. (5.1)].

There have been many attempts to develop a corresponding theory for discrete random variables, often focussing on the Poisson family which is closed under standard integer addition (“Poisson $+$ Poisson $=$ Poisson”). Results in this context include Poisson limit theorems [5, 6], maximum entropy property [7] and monotonicity result [8]. However, [7] and [8] rely on the assumption of ultra-log-concavity (ULC), meaning that they are more restrictive than their Gaussian counterparts.

In this paper we prove new properties of what we refer to as the ‘beamsplitter addition’ $\boxplus_{\eta}$ (see Definition I.1) of random variables supported on the non-negative integers ${\mathbb{Z}}_{+}$. By design, the geometric family is closed under the action of $\boxplus_{\eta}$ (“Geometric $\boxplus_{\eta}$ Geometric $=$ Geometric”). Geometric is more natural than the Poisson since no auxiliary assumptions such as ULC are required to prove maximum entropy.

The beamsplitter addition is motivated by how an optical beamsplitter of transmissivity $\eta\in[0,1]$ “adds” the photon-number distributions of two classical mixtures of number states. It underlies the conjectural Entropy Photon Number Inequality (EPnI) [9, 10], which plays a role analogous to Shannon’s EPI in understanding the capacity of Gaussian bosonic channels. The paper [11] includes a more detailed history of the beamsplitter addition $\boxplus_{\eta}$. The key aim of the main part of the present paper however, is to give a self-contained presentation of the beamsplitter addition $\boxplus_{\eta}$ as a way of combining random variables supported on ${\mathbb{Z}}_{+}$, in a way that is accessible to a purely classical audience. Although some of our results may be known to the quantum information community, our hope is that this paper will stimulate future work by probabilists and classical information theorists on open problems—in particular, a proof of the conjectured EPI and entropic monotonicity under the beamsplitter addition [12].

We first define $\boxplus_{\eta}$ in the notation of [12]. That is:

Definition I.1**.**

Given a random variable supported on ${\mathbb{Z}}_{+}$, define its continuous counterpart ${\mathbf{X}}_{c}$ (a circularly symmetric random variable supported on the complex plane ${\mathbb{C}}$), using a map ${\mathcal{T}}$ with ${\mathbf{X}}_{c}={\mathcal{T}}(X)$ and $X={\mathcal{T}}^{-1}({\mathbf{X}}_{c})$ with actions on mass functions and densities given by [12, Eq. (14),(15)]:

[TABLE]

where ${\mathcal{L}}_{n}$ denotes the $n^{\rm th}$ Laguerre polynomial. As in [12], for $0\leq\eta\leq 1$, we define the beamsplitter addition operation $\boxplus_{\eta}$ acting on random variables supported on ${\mathbb{Z}}_{+}$ by

[TABLE]

where ‘$+$’ on the RHS of (3) denotes standard addition in ${\mathbb{C}}$.

The key contributions and structure of the rest of the paper are as follows. In Section II we define two types of generating functions, for $X$ and ${\mathbf{X}}_{c}$, and prove a new relation between them in Theorem II.4. In Section III, we prove Theorem III.1, which shows that the generating function of $X\boxplus_{\eta}Y$ is a product of generating functions. In Section IV, we show that Theorem III.1 implies a heat equation (Theorem IV.1), which in turn gives a de Bruijn identity (Theorem IV.3). In Section V, we state and prove a new bound on relative entropy under the action of ${\mathcal{T}}$. In Section VI we discuss some future work. The remainder of the main part of the paper contains proofs of its main results. In an Appendix we review necessary ideas from quantum optics in order to show how the main results of the paper may be understood in this context.

II Relation between generating functions

We consider two different kinds of generating functions, exponential and ordinary, recalling that they are related by the Laplace transform (see Lemma VII.1). We write ${\mathbb{E}}\left(X\right)_{(m)}:={\mathbb{E}}X(X-1)\ldots(X-m+1)={\mathbb{E}}X!/(X-m)!$ for the falling moment of a random variable on ${\mathbb{Z}}_{+}$.

Definition II.1**.**

---

Given random variable $X$ with p.m.f. $p_{X}[m]$, $m\in{\mathbb{Z}}_{+}$, consider the sequence ${\mathbb{E}}\left(X\right)_{(m)}/m!$ and write:

1. (a)

The ordinary generating function

[TABLE] 2. (b)

The exponential generating function

[TABLE] 2. 2.

For circularly symmetric ${\mathbf{X}}_{c}$ supported on ${\mathbb{C}}$ with density $p_{{\mathbf{X}}_{c}}$, consider the sequence $\frac{{\mathbb{E}}|{\mathbf{X}}_{c}|^{2m}}{m!}$ and write:

1. (a)

The ordinary generating function

[TABLE] 2. (b)

The exponential generating function

[TABLE]

Note that although (4) and (5) are defined as formal sums, in practice we focus on $t\leq 0$. We first make the following claim, proved in Section VII:

Lemma II.2**.**

For any random variable $X$ supported on ${\mathbb{Z}}_{+}$:

$\widetilde{H}_{X}(t)=\sum_{n=0}^{\infty}p_{X}[n]{\mathcal{L}}_{n}(-t)$, 2. 2.

$p_{X}[m]=\int_{0}^{\infty}\exp(-s)\widetilde{H}_{X}(-s){\mathcal{L}}_{m}(s)ds$.

Example II.3**.**

For geometric $X$ with mean $\lambda$, ${\mathbb{E}}\left(X\right)_{(m)}=m!\lambda^{m}$, so that $H_{X}(t)=1/(1-\lambda t)$ and $\widetilde{H}_{X}(t)=\exp(\lambda t)$. Further ${\mathbf{X}}_{c}$ is circularly symmetric Gaussian with covariance matrix $(1+\lambda)I_{2}/2$, the ${\mathbb{E}}|{\mathbf{X}}_{c}|^{2m}=m!(1+\lambda)^{m}$, so $\phi_{{\mathbf{X}}_{c}}(t)=1/(1-(1+\lambda)t)$ and $\widetilde{\phi}_{{\mathbf{X}}_{c}}(t)=\exp(t(1+\lambda))$.

Example II.3 illustrates the following result, which shows that $X$ and ${\mathbf{X}}_{c}$ have a simple link at the level of their generating functions:

Theorem II.4**.**

For $X$ and ${\mathbf{X}}_{c}$ linked by the transforms (1) and (2) of [12], we can write

[TABLE] 2. 2.

[TABLE]

Proof:

See Section VII. ∎

This result relates the moments of $X$ and ${\mathbf{X}}_{c}$. For brevity, from now on we write $\lambda_{W}$ for the mean of any random variable $W$. Then, for example:

Corollary II.5**.**

The real and imaginary parts of ${\mathbf{X}}_{c}=(X_{1},X_{2})$ have covariance matrix $(1+\lambda_{X}){\mathbf{I}}_{2}/2$, where we write ${\mathbf{I}}_{d}$ for the $d$-dimensional identity matrix.

Proof:

We simply differentiate (9) with respect to $t$ and set $t=0$ to obtain:

[TABLE]

Since ${\mathbf{X}}_{c}$ is circularly symmetric, it is proper (see [14]), and we know ${\mathbb{E}}{\mathbf{X}}_{c}=0$. Further, this means that $(X_{1},X_{2})$ has a covariance matrix which is a multiple of the identity. We deduce that the diagonal entries must equal $(\lambda_{X}+1)/2$. ∎

III Generating functions and beamsplitter addition

We now state the relationship between the generating functions of $X$, $Y$ and $Z=X\boxplus_{\eta}Y$, proved in Section VIII:

Theorem III.1**.**

Given independent random variables $X$ and $Y$ supported on ${\mathbb{Z}}_{+}$, the $Z:=X\boxplus_{\eta}Y$ has generating functions $\widetilde{H}_{Z}$ and $H_{Z}$ satisfying:

[TABLE] 2. 2.

[TABLE]

where we define $M_{X}^{(\eta)}$ and $M_{Y}^{(1-\eta)}$ via the inverse Laplace transform $\left[L^{-1}\cdot\right]$ using the fact that:

[TABLE]

Direct calculation of the derivative of (10), as in Corollary II.5, allows us to deduce that

[TABLE]

Example III.2**.**

If $X$ is geometric with mean $\lambda_{X}$ and $Y$ is geometric with mean $\lambda_{Y}$, using the expressions from Example II.3 and (14) then

The RHS of (10) becomes

[TABLE]

so that $Z=X\boxplus_{\eta}Y$ is geometric with mean $\lambda_{Z}$. 2. 2.

The RHS of (12) is $1/(s-\eta(1+\lambda_{X}))$, so the inverse Laplace transform gives that $M^{(\eta)}_{{\mathbf{X}}_{c}}(t)=\exp(\eta(1+\lambda_{X})t)$, with $M^{(\eta)}_{{\mathbf{Y}}_{c}}(t)=\exp((1-\eta)(1+\lambda_{Y})t)$. As we would expect, this means that $\left(M_{X}^{(\eta)}\times M_{Y}^{(1-\eta)}\right)=\exp((1+\lambda_{Z})t)$ This allows us to deduce that

[TABLE]

and changing variables via $u=1/(s-1)$ we deduce that $H_{Z}(u)=1/(1-\lambda_{Z}u)$ as we would hope.

Remark III.3**.**

Theorem III.1 and Example III.2 suggest the exponential generating function $\widetilde{H}_{X}$ is more amenable than the ordinary generating function $H_{X}$. We state both results for future reference, but recommend the first formulation.

IV De Bruijn identity

Motivated by [1], we give a de Bruijn identity with respect to beamsplitter addition $\boxplus_{\eta}$. The key result is the following discrete analogue of the heat equation, analogous to [7, Corollary 4.2] in the Poisson case:

Theorem IV.1**.**

For a given random variable $X$ consider $Z_{\eta}:=X\boxplus_{\eta}Y$, where $Y$ is geometric. Writing $\lambda(\eta)=\lambda_{Z_{\eta}}=\eta\lambda_{X}+(1-\eta)\lambda_{Y}$ we obtain

[TABLE]

where for any function $u$, we write $\Delta(u[n]):=u[n+1]-u[n]$.

Proof:

See Section IX. ∎

Definition IV.2**.**

For a random variable $X$ with mass function $p_{X}$, define two new p.m.f.s supported on ${\mathbb{Z}}_{+}$ by

[TABLE]

This allows us to deduce the following de Bruijn identity, which is a specialization to number-diagonal states of the more general de Bruijn identity proved by König and Smith [13]:

Theorem IV.3**.**

Given $Z_{\eta}=X\boxplus_{\eta}Y$, where $Y$ is geometric with mean $\lambda_{Y}$, we can write $G_{\eta}$ for a geometric with mean $\lambda(\eta)=\eta\lambda_{X}+(1-\eta)\lambda_{Y}$. Then

[TABLE]

where $p_{Z_{\eta}}^{+}$ and $p_{Z_{\eta}}^{-}$ are defined in terms of (16).

If $X$ is itself geometric then so is $Z_{\eta}$, meaning that $p_{Z_{\eta}}^{+}=p_{Z_{\eta}}^{-}$ (a negative binomial mass function) and the two relative entropy terms on the RHS of Theorem IV.3 vanish as expected.

We focus on the case where $\lambda_{X}=\lambda_{Y}$, where the RHS of Theorem IV.3 becomes a symmetrized relative entropy. If $G$ is geometric with ${\mathbb{E}}G=\lambda_{X}$ then direct calculation gives that

[TABLE]

This allows us to deduce the following log-Sobolev type inequality which may be of independent interest:

Corollary IV.4**.**

For any random variable $X$, if $G$ is geometric with mean ${\mathbb{E}}G=\lambda_{X}$ then:

[TABLE]

Proof:

We consider $Z_{\eta}=X\boxplus_{\eta}Y$, where $Y$ is geometric with $\lambda_{Y}=\lambda_{X}$, and apply [12, Theorem 5], which tells us that $H(Z_{\eta})\geq\eta H(X)+(1-\eta)H(Y)$. Combining this with (17), we can write

[TABLE]

or rearranging that (since $\eta\leq 1$)

[TABLE]

If $\eta\rightarrow 1$, the LHS becomes the derivative $\frac{\partial}{\partial\eta}D(Z_{\eta}\|G)|_{\eta=1}$, and we deduce the result using Theorem IV.3. ∎

In the language of [5], Theorem IV.1 suggests that we can introduce a score function $\rho_{X}$, defined as:

Definition IV.5**.**

For a random variable $X$ with mass function $p_{X}$ and mean $\lambda_{X}$, define a score function

[TABLE]

where we define $\rho_{X}[0]=0$ to ensure $\sum_{n=0}^{\infty}P_{X}[n]\rho_{X}[n]=0$. We define two Fisher-type quantities in terms of it:

[TABLE]

Note that $\rho_{X}$ vanishes if and only if $X$ is geometric, so that $J^{+}(X)$ and $J^{-}(X)$ are $\geq 0$, with equality if and only if $X$ is geometric. Further, if we choose $Y$ to be geometric with mean ${\mathbb{E}}Y=\lambda_{X}$ then (15) becomes

[TABLE]

where as before, we write $\Delta(u[n]):=u[n+1]-u[n]$. Secondly, linearising the logarithm in Corollary IV.4 implies a quadratic version of this result, in the spirit of [15]:

Corollary IV.6**.**

For any random variable $X$, if $G$ is geometric with mean ${\mathbb{E}}G=\lambda_{X}$ then:

[TABLE]

V Log-sum inequality

We state one further result, which controls how the relative entropy behaves under the action of the map ${\mathcal{T}}$.

Theorem V.1**.**

Given two random variables $X$ and $Y$, with ${\mathbf{X}}_{c}={\mathcal{T}}(X)$ and ${\mathbf{Y}}_{c}={\mathcal{T}}(Y)$ then:

[TABLE]

Proof:

Writing $\phi_{n}({\mathbf{r}}):=e^{-|{\mathbf{r}}|^{2}}|{\mathbf{r}}|^{2n}/(n!\pi)$ then, using (1) and the log-sum inequality [16, Theorem 2.7.1], for any ${\mathbf{r}}$:

[TABLE]

Integrating over ${\mathbf{r}}$ we deduce the result since $\int\phi_{n}({\mathbf{r}})d{\mathbf{r}}=1$ for each $n$. ∎

VI Conclusions

We have introduced a new purely classical representation for the beamsplitter addition operation $\boxplus_{\eta}$, with respect to the exponential generating function. We have deduced a heat equation, and recovered a purely classical proof of a special case of the de Bruijn identity of König and Smith [13].

In future work, we hope to use this formalism to prove discrete entropy results based around the geometric family, analogous to the classical results proved for the continuous entropy based around the Gaussian family. In particular, we hope that our results can give insights into a proof of the conjectured discrete EPI under beamsplitter addition [12]—a special case of the Entropy Photon Number Inequality [9, 10]—as well as give insights into convergence to the geometric and the conjectured monotonic increase in entropy under repeated beamsplitter addition [12], analogous to the classical Central Limit Theorem convergence to Gaussians and ‘law of thin numbers’ [6] convergence to Poissons.

VII Proof of Transform relation, Theorem II.4

Proof:

We reverse the order of summation in (5) to obtain

[TABLE]

since ${\mathcal{L}}_{n}(-t)=\sum_{m=0}^{n}\binom{n}{m}\frac{t^{m}}{m!}$ (see [17, Eq. (5.1.6)]). 2. 2.

This result follows on integrating $\exp(-s){\mathcal{L}}_{m}(s)$ times both sides of (22) with $s=-t$, using the orthogonality relation for Laguerre polynomials [17, Eq. (5.1.1)],

[TABLE]

∎

Recall the standard result that the Laplace transform $L$ relates exponential and ordinary generating functions:

Lemma VII.1**.**

Given a sequence ${\mathbf{a}}=(a_{n})_{n=0,1,\ldots}$, if we write $H_{{\mathbf{a}}}(t)=\sum_{n=0}^{\infty}a_{n}t^{n}$ and $\widetilde{H}_{{\mathbf{a}}}(t)=\sum_{n=0}^{\infty}a_{n}t^{n}/n!$, then

[TABLE]

Proof:

This follows since

[TABLE]

∎

Proof:

We can express (7) in terms of the Bessel function $J_{0}$ (see [17, Eq. (1.71.1)]), which we substitute to obtain (25) below. We obtain:

[TABLE]

where (26) follows by substituting (1), (27) follows by moving from Cartesian coordinates $d{\mathbf{r}}$ to polar $rdrd\theta$, (28) uses $u=r^{2}$ and (29) follows by [17, Theorem 5.4.1]. The result follows by Lemma II.2.

Consider taking Laplace transforms of both sides of (8). Using Lemma VII.1 the Laplace transform of the LHS is

[TABLE]

Again by Lemma VII.1, since the Laplace transform of $f(t)\exp(-t)$ is the Laplace transform of $f$ shifted by $1$, the Laplace transform of the RHS is

[TABLE]

Equating (30) and (31), the result follows taking $u=1/t$. ∎

VIII Proof of convolution relation, Theorem III.1

We first state a result that shows how the moments of circularly symmetric random variables on ${\mathbb{C}}$ behave on convolution:

Lemma VIII.1**.**

Given independent circularly symmetric ${\mathbf{X}}_{c}$ and ${\mathbf{Y}}_{c}$ and writing ${\mathbf{Z}}_{c}:=\sqrt{\eta}{\mathbf{X}}_{c}+\sqrt{1-\eta}{\mathbf{Y}}_{c}$, we can write

[TABLE]

Proof:

Consider independent ${\mathbf{U}}_{c}$ and ${\mathbf{V}}_{c}$ and write ${\mathbf{W}}_{c}={\mathbf{U}}_{c}+{\mathbf{V}}_{c}$. Then [18, Eq. (3)] gives

[TABLE]

Multiplying $t^{m}/(m!)^{2}$, and summing, we obtain that

[TABLE]

and the result follows by rescaling. ∎

Putting all this together we obtain:

Proof:

1. This result follows directly on combining (8) and (32).

2. Relabelling $t=1/(s-1)$ in (9) and using Lemma VII.1, for any random variable $U$ we obtain:

[TABLE]

Taking $U=Z$ in (33), and using Lemma VIII.1, we know that $\widetilde{\phi}_{{\mathbf{Z}}_{c}}=M^{(\eta)}_{{\mathbf{X}}_{c}}\times M^{(1-\eta)}_{{\mathbf{Y}}_{c}}$ where $M_{{\mathbf{X}}_{c}}^{(\eta)}(t)=\widetilde{\phi}_{{\mathbf{X}}_{c}}\left(\eta t\right)$. We can use the fact that if $F=\left[Lf\right]$ then $\left[Lf(at)\right](s)=\frac{1}{a}F\left(\frac{s}{a}\right)$ to deduce using (33) that

[TABLE]

and (12) follows. A similar argument based on the fact that $M_{{\mathbf{Y}}_{c}}^{(1-\eta)}(t)=\widetilde{\phi}_{{\mathbf{Y}}_{c}}\left((1-\eta)t\right)$ allows us to deduce (13). ∎

IX Proof of de Bruijn identity, Theorem IV.3

We first prove the heat equation Theorem IV.1:

Proof:

By Lemma II.2.1) we can write

[TABLE]

Using (10) we also write $h(\eta;t)=\widetilde{H}_{X}(\eta t)\exp((1-\eta)\lambda_{Y}t)$ and observe that this satisfies

[TABLE]

Hence, by differentiating (34) and using (35), we obtain

[TABLE]

here (IX) follows using the fact that [17, Eq. (5.1.14)] $z{\mathcal{L}}_{n}^{\prime}(z)=n{\mathcal{L}}_{n}(z)-n{\mathcal{L}}_{n-1}(z)$. and using the three-term relation for Laguerre polynomials, $-z{\mathcal{L}}_{n}(z)=(n+1){\mathcal{L}}_{n+1}(z)-(2n+1){\mathcal{L}}_{n}(z)+n{\mathcal{L}}_{n-1}(z)$ (see [17, Eq. (5.1.10)]). Comparing coefficients of ${\mathcal{L}}_{n}(-t)$ we conclude the result holds. ∎

Proof:

Using (17), and writing $p_{\eta}$ for $p_{Z_{\eta}}$, we can express $D(Z_{\eta}\|G_{\eta})$ as

[TABLE]

For any function $u[n]$, the $\sum_{n=0}^{\infty}\Delta(u[n])\log p_{\eta}[n]=\sum_{n=0}^{\infty}u[n+1]\log(p_{\eta}[n]/p_{\eta}[n+1])$, so (assuming we can exchange the sum and derivative) we obtain

[TABLE]

where (37) follows using (14), and adding factors of $(n+1)$ to the top and bottom of the fraction to deduce the result. ∎

Acknowledgements: The authors thank the organizers of the 2016 Beyond i.i.d. in Information Theory workshop, where discussions leading up to this paper first took seed.

-A Brief review of quantum optics notation

-A1 Bosonic modes, quantum states, characteristic functions

The annihilation operator $a$ of a single mode of an electromagnetic field, and the creation operator $a^{\dagger}$, its Hermitian conjugate, act on the number states $\left\{|n\rangle\right\}$, $n\in{\mathbb{Z}}_{+}$ in the following way:

[TABLE]

with $\langle n|m\rangle=\delta_{nm}$. The number states form a complete orthonormal basis for the infinite-dimensional Hilbert space ${\cal H}$ of a single bosonic mode. A quantum state of a mode $\rho\in{\cal D}({\cal H})$, where ${\cal D}({\cal H})$ is the set of unit-trace positive Hermitian operators (called “density operators”) in ${\cal H}$.

A special class of states called coherent states,

[TABLE]

are eigenstates of $a$, i.e., $a\,|\alpha\rangle=\alpha\,|\alpha\rangle$ for all $\alpha\in{\mathbb{C}}$. Coherent states of a mode form an overcomplete basis, i.e.,

[TABLE]

where ${I}_{\infty}$ is the identity operator in ${\cal H}$. The coherent state $|0\rangle$ is same as the number state $|0\rangle$, known as the vacuum state.

The displacement operator is defined as

[TABLE]

The reason it is called the displacement operator is that it displaces the vacuum state to a coherent state of a given complex amplitude, i.e., $|\alpha\rangle=D(\alpha)\,|0\rangle$, $\forall\alpha\in{\mathbb{C}}$. The matrix elements of $D(\alpha)$ in the number state basis are given by:

[TABLE]

where the latter holds for $m\geq n$. The following is a useful expression:

[TABLE]

The average of an operator $O$ in state $\rho$, is $\langle O\rangle={\rm tr}(\rho O)$. From hereon, we will start labeling the density operator by a subscript (e.g., $X$), where $X$ is the label of the mode whose state we are referring to. We define three characteristic functions of a density operator $\rho_{X}$, with $\zeta\in{\mathbb{C}}$, as follows:

[TABLE]

The Wigner characteristic function is the mean of the displacement operator $D_{X}(\zeta)=e^{-\zeta^{*}a_{X}+\zeta a_{X}^{\dagger}}$. It is simple to show that for any state $\rho_{X}$, its characteristic functions are related by:

[TABLE]

The Husimi function, popularly known as the $Q$-function, of a state $\rho_{X}$ is defined as:

[TABLE]

It readily follows from (41) that $Q_{X}(\alpha)$ is a proper probability distribution, i.e., $0\leq Q_{X}(\alpha)<1/\pi$ and $\int_{\mathbb{C}}Q_{X}(\alpha)d^{2}\alpha=1$ for any state $\rho_{X}$.

The $Q$-function and the anti-normal ordered characteristic function are related by a Fourier transform relationship:

[TABLE]

The density operator can in turn be retrieved from the characteristic functions as follows:

[TABLE]

It follows from Eqs. (43),  (50) and the fact that ${\cal L}(-t)=\sum_{m=0}^{n}\binom{n}{m}\frac{t^{m}}{m!}$, that the normal ordered characteristic function of the number state $|n\rangle$, i.e., $\rho_{X}=|n\rangle\langle n|$ is given by,

[TABLE]

-A2 Input output relationship of a beamsplitter

The Heisenberg description of the input-output relationship of a beamsplitter of transmissivity $\eta$ is given by:

[TABLE]

where $a_{X}$, $a_{Y}$ and $a_{Z}$ are annihilation operators of two input modes (X and Y) and an output mode (Z), respectively. We take $\rho_{X}$, $\rho_{Y}$ and $\rho_{Z}$ to be the density operators of the quantum states of the respective modes. Let us also assume the input states are statistically independent (viz., in a product state), i.e., $\rho_{XY}=\rho_{X}\otimes\rho_{Y}$.

It readily follows from (59), and the fact that $\rho_{X}$ and $\rho_{Y}$ are statistically independent, that the characteristic functions undergo a scaled multiplication under the beamsplitter mixing.

[TABLE]

This is reminiscent of how the probability distribution functions of statistically-independent random variables $X$ and $Y$ undergo a scaled convolution under the operation $Z=\sqrt{\eta}X+\sqrt{1-\eta}Y$, and that the Fourier transforms of the pdfs undergo a scaled multiplication.

-A3 Number diagonal states

Let us consider a state $\rho_{X}=\sum_{n=0}^{\infty}p_{X}[n]|n\rangle\langle n|$, which is diagonal in the number basis. The unit trace condition of a density operator, ${\rm tr}(\rho_{X})=1$, implies that $\left\{p_{X}[n]\right\}$, $n\in{\mathbb{Z}}_{+}$ is a proper p.m.f., where ${\mathbb{Z}}_{+}$ is the set of non-negative integers. We interpret $p_{X}[n]$ as the p.m.f. of a discrete-valued random variable $X$ that takes values in ${\mathbb{Z}}_{+}$. The Q-function $Q_{X}(\alpha)$ is circularly symmetric (i.e., only a function of the magnitude $|\alpha|$) if and only if $\rho_{X}$ is number diagonal [CITE]. Interpreting $Q_{X}(\alpha)$ as the p.d.f. of a circularly-symmetric continuous-valued random variable ${\boldsymbol{X}}_{c}$, simple algebra leads us to the following $1$-to-$1$ relationship between the p.d.f. of ${\boldsymbol{X}}_{c}$ and the p.m.f. of $X$:

[TABLE]

which was introduced in [12] as the pair of maps, ${\mathbf{X}}_{c}={\mathcal{T}}(X)$ and $X={\mathcal{T}}^{-1}({\mathbf{X}}_{c})$.

If two number diagonal states $\rho_{X}=\sum_{n=0}^{\infty}p_{X}[n]|n\rangle\langle n|$ and $\rho_{Y}=\sum_{n=0}^{\infty}p_{Y}[n]|n\rangle\langle n|$ interfere on a beamsplitter of transmissivity $\eta$ (59), the output state is also number diagonal, $\rho_{Z}=\sum_{n=0}^{\infty}p_{Z}[n]|n\rangle\langle n|$. We define ${\boldsymbol{Y}}_{c}$ and ${\boldsymbol{Z}}_{c}$ analogously from using (1). Since the $Q$-functions $p_{{\boldsymbol{X}}_{c}}({\boldsymbol{r}})$, $p_{{\boldsymbol{Y}}_{c}}({\boldsymbol{r}})$ and $p_{{\boldsymbol{Z}}_{c}}({\boldsymbol{r}})$ are Fourier inverses of the respective antinormal-ordered characteristic functions $\chi_{A}^{\rho_{X}}(\zeta)$, $\chi_{A}^{\rho_{Y}}(\zeta)$ and $\chi_{A}^{\rho_{Z}}(\zeta)$, given the characteristic functions multiply as in (61), we conclude that ${\boldsymbol{Z}}_{c}=\sqrt{\eta}{\boldsymbol{X}}_{c}+\sqrt{1-\eta}\,{\boldsymbol{Y}}_{c}$. As in [12], for $0\leq\eta\leq 1$, we define the beamsplitter addition operation $\boxplus_{\eta}$ acting on random variables supported on ${\mathbb{Z}}_{+}$ by

[TABLE]

where the ‘$+$’ on the RHS of (65) is standard addition in ${\mathbb{C}}$.

-B Transform relations involving discrete and continuous exponential generating functions

Let us consider a discrete-valued random variable $X\in{\mathbb{Z}}_{+}$, a corresponding circularly-symmetric continuous-valued random variable ${\mathbf{X}}_{c}\in{\mathbb{C}}$, and a number-diagonal state $\rho_{X}$ with photon number distribution $p_{X}$, as described in Section -A3. Let us evaluate $\chi_{N}^{\rho_{X}}(\zeta)$ using the definition (46) and the normal ordered characteristic function of a number state (58):

[TABLE]

Therefore, $\chi_{N}^{\rho_{X}}(\zeta)$ is nothing but the exponential generating function of $X$ (see (5))

[TABLE]

evaluated at $t=-|\zeta|^{2}$. Here we wrote ${\mathbb{E}}\left(X\right)_{(m)}:={\mathbb{E}}X(X-1)\ldots(X-m+1)={\mathbb{E}}X!/(X-m)!$ for the falling moment of a random variable on ${\mathbb{Z}}_{+}$. In other words,

[TABLE]

Using Eqs. (57) and (69), we get

[TABLE]

Next, we evaluate the number-basis diagonal elements of $\rho_{X}$ using Eq. (56), and use (43) to obtain:

[TABLE]

where in the last step we expressed $\zeta=re^{i\theta}$, used the substitution $|\zeta|^{2}=r^{2}=u$, $d^{2}\zeta=2dr\,d\theta$, integrated over $\theta\in(0,2\pi]$, and used the relationship (69).

Hence we have the following (Lemma II.2 of the main text):

Lemma .1**.**

For any random variable $X$:

$\widetilde{H}_{X}(t)=\sum_{n=0}^{\infty}p_{X}[n]{\mathcal{L}}_{n}(-t)$, for $t\in(-\infty,0]$, and 2. 2.

$p_{X}[m]=\int_{0}^{\infty}\exp(-u)\widetilde{H}_{X}(-u){\mathcal{L}}_{m}(u)du$.

Now, let us consider the exponential generating function of (7), that is

[TABLE]

It can be shown by evaluating Eq. (52) for circularly-symmetric $Q_{X}(\alpha)$, that

[TABLE]

Using $\chi_{A}^{\rho_{X}}(\zeta)=e^{-|\zeta|^{2}}\chi_{N}^{\rho_{X}}(\zeta)$ (see (49) and (50)), we get the following theorem (Theorem II.4 of the main text):

Theorem .2**.**

For $X$ and ${\mathbf{X}}_{c}$ linked by the transforms (1) and (2) of [12], we can write

[TABLE]

Finally, since we interpreted $\widetilde{\phi}_{{\mathbf{X}}_{c}}(t)$ and $\widetilde{H}_{X}(t)$ as characteristic functions, using Eqs. (60) and (61), we obtain the following (Theorem III.1 and Lemma VIII.1 of the main text, respectively):

[TABLE]

-C The discrete variable heat equation

A bosonic Gaussian channel (BGC) can be characterized by its action on the characteristic function of the input state. In particular, for one-mode gauge-covariant BGCs, the transformation of the input state $\rho_{\rm in}$ to the output state $\rho_{\rm out}$, expressed in terms of their characteristic functions assumes the following form (we are using the Wigner characteristic functions below):

[TABLE]

where $\tau>0$ is the loss or gain parameter and $y$ parametrizes added (Gaussian) noise. The BGC is a valid TPCP map if $y\geq|\tau-1|$. For a lossy bosonic channel of transmissivity $\eta\in(0,1)$ and added thermal noise of mean photon number $N$, $\tau=\eta$ and $y=(1-\eta)(2N+1)$. For a phase-insensitive bosonic amplifier of gain $\kappa>1$ and added thermal noise of mean photon number $N$, $\tau=\kappa$ and $y=(\kappa-1)(2N+1)$. For a unit-gain additive thermal noise channel with photon-number-unit noise variance $N$, $\tau=1$ and $y=2N$. All single-mode gauge-covariant BGCs posses a semi-group structure, and consequently that a gauge-covariant single-mode BGC can be represented as a one-parameter linear TPCP map [19],

[TABLE]

with manifest semi-group structure

[TABLE]

Here $\L$ is a Lindblad operator that generates the dynamics of a gauge-covariant BGC, and $t$ can be viewed as a time parameter corresponding to a continuous action of the channel on the input state $\rho(0)$, resulting in the final state $\rho(t)$ at time $t>0$. With that interpretation, the equation of motion for $\rho(t)$, under the action of the channel, is given by

[TABLE]

For Gauge-covariant BGCs, the Lindblad operator is given by

[TABLE]

where

[TABLE]

For a lossy channel with additive thermal noise, $\gamma_{+}=N,\gamma_{-}=N+1$, where $N$ is the mean photon number of the thermal state. The channel transmissivity $\eta=e^{-t}$. For a phase-insensitive noisy amplifier channel, $\gamma_{+}=N+1,\gamma_{-}=N$, and the amplifier gain $\kappa=e^{t}$. Finally, for an additive Gaussian noise channel, $\gamma_{+}=\gamma_{-}=1$ with $N=t$.

When $\rho_{X}$ is number diagonal, and $\rho_{Y}$ a number diagonal with geometric $p_{Y}[n]$ (thermal state) of mean $\lambda_{Y}$, it is simple to verify that  (81) reduces to the following (Theorem IV.1 of the main text):

Theorem .3**.**

For a given random variable $X$ consider $Z_{\eta}:=X\boxplus_{\eta}Y$, where $Y$ is a geometric with mean $\lambda_{Y}$. Writing $\lambda(\eta)=\eta\lambda_{X}+(1-\eta)\lambda_{Y}$ for the mean of $Z_{\eta}$ we obtain

[TABLE]

where for any function $u$, we write $\Delta(u[n]):=u[n+1]-u[n]$.

-D Log-sum inequality as monotonicity of quantum relative entropy under the action of a TPCP map

Given two quantum states $\rho,\sigma\in{\cal D}({\cal H}_{1})$, and a quantum channel—a trace-preserving completely positive (TPCP) map—$T:{\cal D}({\cal H}_{1})\to{\cal D}({\cal H}_{2})$, we have that [20]:

[TABLE]

where $D(\rho||\sigma)={\rm Tr}\left\{\rho\left(\log\rho-\log\sigma\right)\right\}$ is the quantum relative entropy between the states $\rho$ and $\sigma$. If $\rho$ and $\sigma$ are simultaneously diagonalizable, i.e., $\rho=\sum_{k}p[k]|k\rangle\langle k|$ and $\sigma=\sum_{k}q[k]|k\rangle\langle k|$ for some complete orthonormal basis $\left\{|k\rangle\right\}$ in ${\cal H}_{1}$, then the quantum relative entropy equals the classical relative entropy between the probability distributions in the common spectral basis, i.e., $D(\rho||\sigma)=D(p||q)$. Eq. (86) states that the action of a channel cannot increase the quantum relative entropy between a pair of input states.

Any quantum measurement can be interpreted as a (measure-and-prepare) quantum channel. Therefore, if a measurement, described by POVM operators $\left\{E_{k}\right\}$, act on states $\rho$ and $\sigma$ to induce probability distributions $p[k]$ and $q[k]$ respectively, i.e., ${\rm tr}(\rho E_{k})=p[k]$ and ${\rm tr}(\sigma E_{k})=q[k]$, (86) reduces to $D(\rho||\sigma)\geq D(p||q)$.

Consider number diagonal states $\rho_{X}=\sum_{n=0}^{\infty}p_{X}[n]|n\rangle\langle n|$ and $\rho_{Y}=\sum_{n=0}^{\infty}p_{Y}[n]|n\rangle\langle n|$. Since $\rho_{X}$ and $\rho_{Y}$ are both diagonal in the number basis, $D(\rho_{X}||\rho_{Y})=D(X||Y)$, where the RHS is the classical relative entropy between random variables $X$ and $Y$ whose p.m.f.s are $p_{X}$ and $p_{Y}$ respectively. Let us consider measuring $\rho_{X}$ and $\rho_{Y}$ in the coherent-state (overcomplete) basis $\left\{\left(|\alpha\rangle\langle\alpha|\right)/\pi\right\},\alpha\in{\mathbb{C}}$. See (41). The measurement by definition induces the continuous circularly-symmetric distributions $p_{{\boldsymbol{X}}_{c}}$ and $p_{{\boldsymbol{Y}}_{c}}$, the respective Husimi functions of $\rho_{X}$ and $\rho_{Y}$. Therefore, the statement of Theorem V.1 follows.