Canonical divergence for measuring classical and quantum complexity

TL;DR

This paper introduces a new canonical divergence that generalizes classical and quantum complexity measures, unifying them with known divergences like Kullback-Leibler and quantum relative entropy.

Contribution

It proposes a unified divergence measure applicable to both classical probability measures and quantum states, extending information geometry tools.

Findings

Divergence coincides with Kullback-Leibler on probability simplices.

Reduces to quantum relative entropy on density operators.

Provides a unified framework for classical and quantum complexity measures.

Abstract

A new canonical divergence is put forward for generalizing an information-geometric measure of complexity for both, classical and quantum systems. On the simplex of probability measures it is proved that the new divergence coincides with the Kullback-Leibler divergence, which is used to quantify how much a probability measure deviates from the non-interacting states that are modeled by exponential families of probabilities. On the space of positive density operators, we prove that the same divergence reduces to the quantum relative entropy, which quantifies many-party correlations of a quantum state from a Gibbs family.

Equations

$$\mathrm{P}(S):=(\Sigma \circ \Pi )(S)$$

$$\mathcal{D}(S,S^{\prime })\ge 0,\mathcal{D}(S,S^{\prime })=0\text{mbox}iffS=S^{\prime }.$$

$$\mathrm{C}(S):=\mathcal{D}(S,\mathrm{P}(S)).$$

$$\mathrm{C}(S)=\mathcal{D}(S,\mathrm{P}(S))=\underset{S^{\prime }\in \mathcal{N}}{\inf }\mathcal{D}(S,S^{\prime }).$$

$$X\mathrm{g}\left(Y,Z\right)=\mathrm{g}\left({\nabla }_{X}Y,Z\right)+\mathrm{g}\left(Y,{\nabla }_X^{\ast }Z\right),\forall X,Y,Z\in \mathcal{T}(\mathrm{M}),$$

$${\mathrm{g}}_{ij}(p)=-{{\partial }_i{\partial }_j^{\prime }\mathcal{D}({\mathbf{\xi }}_p,{\mathbf{\xi }}_q)|}_{p=q}={{\partial }_i^{\prime }{\partial }_j^{\prime }\mathcal{D}({\mathbf{\xi }}_p,{\mathbf{\xi }}_q)|}_{p=q}$$

$${\Gamma }_{ijk}(p)=-{{\partial }_i{\partial }_j{\partial }_k^{\prime }\mathcal{D}({\mathbf{\xi }}_p,{\mathbf{\xi }}_q)|}_{p=q},{\Gamma }_{ijk}^{\ast }(p)=-{{\partial }_i^{\prime }{\partial }_j^{\prime }{\partial }_k\mathcal{D}({\mathbf{\xi }}_p,{\mathbf{\xi }}_q)|}_{p=q},$$

$${\partial }_i=\frac{\partial }{\partial {\xi }_p^i}\text{mbox}and{\partial }_i^{\prime }=\frac{\partial }{\partial {\xi }_q^i}$$

$$\mathcal{D}(p,q):={\int }_0^1{⟨{\mathrm{X}}_t(p),\dot{\tilde{\sigma }}(t)⟩}_{\tilde{\sigma }(t)}dt,{\mathrm{X}}_t(p):={\exp }_{\tilde{\sigma }(t)}^{-1}(p).$$

$$\mathcal{D}(p,q)={\int }_0^{1}t\Vert \dot{\sigma }(t){\Vert }^{2}dt.$$

$${\mathcal{D}}^{\ast }(p,q)={\int }_0^{1}t\Vert {\dot{\sigma }}^{\ast }(t){\Vert }^{2}dt,$$

$$\mathrm{KL}(p,q):=\sum _{i=1}^{n+1}{p}_i\log \left(\frac{p_i}{q_i}\right)$$

$$\mathrm{KL}\left(p,{\mathcal{E}}_{\mathfrak{S}}\right):=\underset{q\in {\mathcal{E}}_{\mathfrak{S}}}{\inf }\mathrm{KL}(p,q)=\mathrm{KL}(p,\hat{p}).$$

$$\mathrm{KL}\left(p,{\mathcal{E}}_{\mathfrak{S}}\right)=\mathrm{KL}\left(p,p_1\otimes \dots \otimes p_n\right)=\sum _{i}H(X_i)-H(X_1,\dots ,X_n),$$

$$\mathrm{Q}(\rho ,{\mathcal{E}}_k):=\underset{\sigma \in {\mathcal{E}}_k}{\inf }\mathrm{Q}(\rho ,\sigma )$$

$$\mathrm{Q}(\rho ,\sigma )=\mathrm{Tr}\rho \left(\log \rho -\log \sigma \right),$$

$$\mathrm{Tr}\left({\sigma }_{i}a\right)=\mathrm{Tr}\left(\rho (a\otimes {\mathbb{I}}_{[n]\backslash \{i\}})\right),$$

$$\mathrm{Q}(\rho ,{\mathcal{E}}_1)=\sum _i\tilde{H}({\sigma }_i)-\tilde{H}(\rho ),$$

$${\mathcal{S}}_n:=\left\{p=\sum _i{p}_i{\delta }^i\in {\mathbb{R}}^n|p_i>0\text{mbox}foralli,\text{mbox}and\sum _i{p}_i=1\right\}.$$

$${\mathrm{g}}_p(X,Y):=\sum _i\frac{1}{p_i}{X}^i{Y}^i,X,Y\in {\mathrm{T}}_p{\mathcal{S}}_n.$$

$${\gamma }_m(t)=p+t(q-p),t\in [0,1]$$

$${\gamma }_e(t)=\sum _i\frac{p_i{\left(\frac{q_i}{p_i}\right)}^t}{{\sum }_j{p}_j{\left(\frac{q_j}{p_j}\right)}^t}{\delta }^i,t\in [0,1].$$

$$\mathcal{D}(p,q)$$

$${\mathcal{D}}^{\ast }(p,q)$$

$${\gamma }_e^i(t)=\frac{p_i{\left(\frac{q_i}{p_i}\right)}^t}{{\sum }_j{p}_j{\left(\frac{q_j}{p_j}\right)}^t}.$$

$${\dot{\gamma }}_e^i(t)$$

$${\mathcal{D}}^{\ast }(p,q)$$

$$\frac{{\dot{\gamma }}_e^i(t)}{{\gamma }_e^i(t)}=\left(\log \frac{q_i}{p_i}-\frac{\mathrm{d}}{\mathrm{d}t}\log \sum _j{p}_j{\left(\frac{q_j}{p_j}\right)}^t\right).$$

$$\frac{\mathrm{d}}{\mathrm{d}t}\log \sum _j{p}_j{\left(\frac{q_j}{p_j}\right)}^t=\frac{{\sum }_j{p}_j{\left(\frac{q_j}{p_j}\right)}^t\log \frac{q_j}{p_j}}{{\sum }_j{p}_j{\left(\frac{q_j}{p_j}\right)}^t},$$

$${\mathcal{D}}^{\ast }(p,q)$$

$${\mathcal{D}}^{\ast }(p,q)=\mathrm{KL}(q,p)=\mathcal{D}(q,p).$$

Full text

I Introduction

The many fields of applicability of methods of information geometry to the science of complexity encompass both classical and quantum systems FeliceChaos . Among them, an information-geometric approach to the complexity as the extent to which an object, as a whole, is more than its parts was established in Aycomplexity and then developed to relate various known measures of complexity to a general class of information-geometric complexity measures (see Ay17 for a comprehensive overview on this topic). The general idea for quantifying the extent to which the system is more than the sum of its parts is the following. Let $\displaystyle\mathcal{S}$ be a set of systems; for any system $\displaystyle S\in\mathcal{S}$, we assign the collection of system parts which may be an element of a set $\displaystyle\mathcal{S}_{0}$ that formally differs from $\displaystyle\mathcal{S}$. The corresponding assignment $\displaystyle\Pi:\mathcal{S}\rightarrow\mathcal{S}_{0}$ can be interpreted as a reduced description of the system $\displaystyle S$ in terms of its parts. Having the parts $\displaystyle\Pi(S)$, we have to reconstruct $\displaystyle S$ by taking the sum of the parts in order to obtain a system that can be compared with the original system. The corresponding construction map is denoted by $\displaystyle\Sigma:\mathcal{S}_{0}\rightarrow\mathcal{S}$. The composition

[TABLE]

then corresponds to the sum of parts of the system $\displaystyle S$, and we can compare $\displaystyle S$ with $\displaystyle\mathrm{P}(S)$. It turns out that $\displaystyle\mathrm{P}$, under natural conditions, is the projection $\displaystyle\mathrm{P}:\mathcal{S}\rightarrow\mathcal{N}$ to the set of non-complex systems $\displaystyle\mathcal{N}:=\{S\in\mathcal{S}\,|\,\mathrm{P}(S)=S\}$ Ay11 . Therefore, the quantification of how much the system $\displaystyle S$ differs from $\displaystyle\mathrm{P}(S)$ is established by a divergence function $\displaystyle\mathcal{D}:\mathcal{S}\times\mathcal{S}\rightarrow\mathbb{R}$ such that

[TABLE]

Finally, the complexity of a system $\displaystyle S$ is defined by

[TABLE]

Clearly, there are many choices for the divergence $\displaystyle\mathcal{D}$, thus such a complexity measure is far from being unique. However, to ensure compatibility with $\displaystyle\mathrm{P}$, one has to further assume that $\displaystyle\mathcal{D}$ satisfies

[TABLE]

Here comes the role of a canonical divergence for providing an information-geometric measure of complexity which can be interpreted as unique.

In the framework of information geometry, a dual structure $\displaystyle({\rm g},\nabla,\nabla^{*})$ on a smooth manifold $\displaystyle\mathrm{M}$ is given in terms of a metric tensor and two affine connections, which are dual in the following sense Amari00 :

[TABLE]

where $\displaystyle\mathcal{T}(\mathrm{M})$ denotes the space of sections on $\displaystyle\mathrm{M}$. Eguchi named a function $\displaystyle\mathcal{D}:\mathrm{M}\times\mathrm{M}\rightarrow\mathbb{R}$ satisfying the property in Equation (1) as a contrast (or divergence) function whenever $\displaystyle\mathcal{D}$ allows recovering the dual structure $\displaystyle({\rm g},\nabla,\nabla^{*})$ on $\displaystyle\mathrm{M}$ in the following way Eguchi85 :

[TABLE]

where

[TABLE]

and $\displaystyle\{\boldsymbol{\xi}_{p}:=(\xi_{p}^{1},\ldots,\xi_{p}^{n})\}$ and $\displaystyle\{\boldsymbol{\xi}_{q}:=(\xi_{q}^{1},\ldots,\xi_{q}^{n})\}$ are local coordinate systems of $\displaystyle p$ and $\displaystyle q$, respectively. Here, $\displaystyle\Gamma_{ijk}={\rm g}\left(\nabla_{\partial_{i}}\partial_{j},\partial_{k}\right)$ and $\displaystyle\Gamma^{*}_{ijk}={\rm g}\left(\nabla^{*}_{\partial_{i}}\partial_{j},\partial_{k}\right)$ are the connection symbols of $\displaystyle\nabla$ and $\displaystyle\nabla^{*}$, respectively. The investigation on a divergence function allowing to recover the dualistic structure on a smooth manifold is usually referred to as the inverse problem in information geometry. Matumoto matumoto1993 showed that such divergence exists for any statistical manifold. However, it is not unique and there are infinitely many divergences that give the same dual structure. Hence, the search for a divergence that can be somehow considered as the most natural is of upmost importance. When a manifold is dually flat, Amari and Nagaoka Amari00 introduced a Bregman type divergence to this end, with relevant properties concerning the generalized Pythagorean theorem and the geodesic projection theorem.This is referred to as canonical divergence and it is commonly assessed as the natural solution of the inverse problem in information geometry for dually flat manifolds. However, the need for a general canonical divergence, which applies to any dualistic structure, is a very crucial issue, as pointed out in AyTusch . In any case, such a divergence should recover the canonical divergence of Bregman type if applied to a dually flat structure. In addition, in the self-dual case where $\displaystyle\nabla=\nabla^{*}$ coincides with the Levi–Civita connection of $\displaystyle{\rm g}$, the divergence $\displaystyle\mathcal{D}$ should be one half of the squared Riemannian distance: $\displaystyle\mathcal{D}(p,q)=\frac{1}{2}\,d(p,q)^{2}$ Ay17 . In the context of the information-geometric approach to complexity, a further requirement is needed to ensure the compatibility in Equation (3). This is the geodesic projection property, which, in the present context, states that every minimizer $\displaystyle\mathrm{P}(S)$ of $\displaystyle\mathcal{D}$ is achieved by the geodesic projection of $\displaystyle S$ onto the set of non-complex systems. In Ay15 , Ay and Amari recently introduced a canonical divergence that satisfies all these requirements. Such a divergence is defined in terms of geodesic integration of the inverse exponential map. More precisely, given $\displaystyle p,q\in\mathrm{M}$ and the $\displaystyle\nabla$-geodesic $\displaystyle\widetilde{\sigma}(t)\,(0\leq t\leq 1)$ connecting $\displaystyle q$ with $\displaystyle p$, the canonical divergence introduced in Ay15 is given by

[TABLE]

Here, $\displaystyle\exp:{\rm T}\mathrm{M}\rightarrow\mathrm{M}$ denotes the exponential map of $\displaystyle\nabla$, which is defined by $\displaystyle\exp(X)=\sigma_{X}(1)$ whenever the $\displaystyle\nabla$-geodesic $\displaystyle\sigma_{X}(t)$, satisfying $\displaystyle\dot{\sigma}_{X}(0)=X$, exists on an interval of $\displaystyle t$ containing $\displaystyle[0,1]$. Therefore, if $\displaystyle\sigma(t)\,(0\leq t\leq 1)$ is the $\displaystyle\nabla$-geodesic such that $\displaystyle\sigma(0)=p$ and $\displaystyle\sigma(1)=q$, then $\displaystyle\exp_{p}^{-1}(q):=\dot{\sigma}(0)$. According to this definition, we have that $\displaystyle\mathrm{X}_{t}(p)={\rm P}_{\sigma(t)}\,\mathrm{X}_{p}(\sigma(t))=t\,\dot{\sigma}(t)$, where $\displaystyle{\rm P}$ is the $\displaystyle\nabla$-parallel transport from $\displaystyle p$ to $\displaystyle\sigma(t)$. This implies that the divergence $\displaystyle\mathcal{D}(p,q)$ assumes the following useful expression:

[TABLE]

Analogously, the dual function of $\displaystyle\mathcal{D}(p,q)$ is defined as the $\displaystyle\nabla^{*}$-geodesic integration of the inverse of the $\displaystyle\nabla^{*}$-exponential map Ay15 . Therefore, we have for the dual divergence $\displaystyle\mathcal{D}^{*}$ a similar expression as Equation (7) for the canonical divergence $\displaystyle\mathcal{D}$:

[TABLE]

where $\displaystyle\sigma^{*}(t)\,(0\leq t\leq 1)$ is the $\displaystyle\nabla^{*}$-geodesic connecting $\displaystyle p$ with $\displaystyle q$. Therefore, the compatibility in Equation (3) of $\displaystyle\mathcal{D}$ with $\displaystyle\mathrm{P}$ suggests that the projection $\displaystyle\mathrm{P}(S)$ of a system $\displaystyle S$ onto the space of non-complex systems can be achieved along the geodesic connection $\displaystyle S$ with $\displaystyle\mathrm{P}(S)$. Actually, it has recently been proved that the $\displaystyle\nabla$-geodesic minimizes the action integral of a suitably chosen kinetic energy HJ2 . An analogous result holds about the $\displaystyle\nabla^{*}$-geodesic. In this way, both divergences, $\displaystyle\mathcal{D}(p,q)$ and $\displaystyle\mathcal{D}^{*}(p,q)$, turn out to solve the Hamilton–Jacobi problem in information geometry, as put forward in HJ1 .

The search for a general canonical divergence is still an open problem and it turns out to be of upmost importance in the context of the information-geometric approach to complexity (see progresses along this avenue put forward in Ay15 ; Felice18 ).

In this article, we aim to propose the canonical divergence in Equation (7) as an efficient tool for providing a unified definition of complexity measures. For this reason, we firstly consider $\displaystyle\mathcal{D}$ on the simplex of probability distributions where a measure of complexity as one instance of Equation (2) is supplied in terms of the Kullback–Leibler (KL)-divergence Ay11 .

The general methods described for defining the complexity measure in Equation (2) can be particularized to the systems consisting of a finite node set $\displaystyle V$ and each node $\displaystyle v\in V$ can be in finitely many states $\displaystyle I_{v}$. Then, we model the whole system as a probability measure $\displaystyle p$ on the corresponding product configuration set $\displaystyle I_{V}=\prod_{v\in V}I_{v}$. The parts are given by marginals $\displaystyle p_{A}$ where $\displaystyle A$ is taken from a set of subsets of $\displaystyle V$, denoted by $\displaystyle\mathfrak{S}$. Therefore, the decomposition map $\displaystyle\Pi$ reads in this case as $\displaystyle\Pi(p)=\left(p_{A}\right)_{A\in\mathfrak{S}}$, whereas the reconstruction map $\displaystyle\Sigma$ is defined by the maximum entropy estimate $\displaystyle\hat{p}$ of $\displaystyle p$, leading to the projection $\displaystyle\pi_{\mathfrak{S}}:p\mapsto\hat{p}$. The image of $\displaystyle\pi_{\mathfrak{S}}$ turns out to be the closure of an exponential family $\displaystyle\mathcal{E}_{\mathfrak{S}}$, which plays the role of the set $\displaystyle\mathcal{N}$ of non-complex systems. A deviation measure, which is compatible with the maximum entropy projection $\displaystyle\pi_{\mathfrak{S}}$ is then the (KL)-divergence, which is defined by

[TABLE]

on the $\displaystyle n$-simplex $\displaystyle\mathcal{P}_{n}=\{p=(p_{1},\ldots,p_{n})\,|\,p_{i}>0\,,\sum_{i}p_{i}=1\}$ Eguchi85 . Finally, the measure of complexity as one instance of Equation (2) is obtained by

[TABLE]

We may notice that, if $\displaystyle\mathfrak{S}$ consists of all subsets of $\displaystyle V$ of cardinality $\displaystyle 1$, elements of the set $\displaystyle\mathcal{E}_{\mathfrak{S}}$ of non-complex systems are totally uncorrelated in the sense that $\displaystyle q\in\mathcal{E}_{\mathfrak{S}}$ has the product form $\displaystyle q=q_{1}\otimes\ldots\otimes q_{n}$ Aycomplexity . Consider random variables $\displaystyle X_{1},\ldots,X_{n}$ with joint probability distribution $\displaystyle p$ and marginal probability distributions $\displaystyle p_{1},\ldots,p_{n}$. Then, we have

[TABLE]

where $\displaystyle H$ is the Shannon entropy. This quantity is referred to as multi information and denoted by $\displaystyle I(X_{1},\ldots,X_{n})$. In particular, when $\displaystyle n=2$, this is nothing but the mutual information. Very remarkably, the minimizer $\displaystyle\hat{p}$ in the closure of $\displaystyle\mathcal{E}_{\mathfrak{S}}$ of the (KL)-divergence, namely $\displaystyle\mathrm{KL}(p,\hat{p})=\inf_{q\in\mathcal{E}_{\mathfrak{S}}}\mathrm{KL}(p,q)$, is obtained by projecting $\displaystyle p$ onto the closure of $\displaystyle\mathcal{E}_{\mathfrak{S}}$ along a mixture $\displaystyle(m)$-geodesic Eguchi92 . This is usually referred to as the geodesic projection property of the (KL)-divergence. The geometric structure given by the Fisher metric, the mixture $\displaystyle(m)$ and exponential $\displaystyle(e)$ affine connections was introduced by Amari and Nagaoka on the space of probability densities for studying statistical estimation problems Amari00 .

In this article, we then consider both divergences, $\displaystyle\mathcal{D}$ and $\displaystyle\mathcal{D}^{*}$, on $\displaystyle\mathcal{P}_{n}$ with the endowed dualistic structure given by the classic Fisher metric and the mixture $\displaystyle(m)$ and the exponential $\displaystyle(e)$ connections. Here, we show that $\displaystyle\mathcal{D}(q,p)=\mathrm{KL}(q,p)=\mathcal{D}^{*}(p,q)$. Actually, this result has already been shown in Ay15 . However, we prove it differently by relying on the nice representations of $\displaystyle\mathcal{D}$ and $\displaystyle\mathcal{D}^{*}$ given by Equations (7) and (8), respectively. This proves that $\displaystyle\mathcal{D}$ can be interpreted as a generalization of the (KL)-divergence.

A further step for proving the effectiveness of $\displaystyle\mathcal{D}$ is to consider it (and its dual function) on the manifold of finite quantum states where the general idea for defining a complexity measure of a classic system expressed by Equation (2) has been extended to the quantum setting in terms of the quantum relative entropy Weis15 . More precisely, by considering a composite set of $\displaystyle n\in\mathbb{N}$ units (parties, particles), $\displaystyle[n]:=\{1,\ldots,n\}$, the composite system is described by the product algebra $\displaystyle\mathcal{A}_{[n]}:=\mathcal{A}_{1}\otimes\ldots\otimes\mathcal{A}_{n}$. Here, $\displaystyle\mathcal{A}_{i}\subset M_{n_{i}}$ is the $\displaystyle C^{*}$-subalgebra of complex $\displaystyle n_{i}\times n_{i}$ matrices such that the identity $\displaystyle\mathbb{I}_{n_{i}}\in\mathcal{A}_{i}$. The many-party correlations are quantified in the state of a composite quantum system which can not be observed in subsystems composed of less than a given number of parties. In this context, the exponential families, which amount to the non-complex system in the classical case, are replaced by states that are fully described by their restriction to selected subsystems. These correspond to the family of Gibbs states $\displaystyle\mathcal{E}_{k}:=\{e^{H_{k}}/{\rm Tr}e^{H_{k}}\}$ of the $\displaystyle k$-local Hamiltonians $\displaystyle H_{k}$. Here, a $\displaystyle k$-local Hamiltonian is defined as a sum of product terms $\displaystyle a_{1}\otimes\ldots\otimes a_{n}$ with at most $\displaystyle k$-non-scalar factors $\displaystyle a_{i}$, where $\displaystyle a_{i}$ denotes a real self-adjoint operator. Therefore, the many-party correlations of a composite quantum state $\displaystyle\rho\in\mathcal{A}_{[n]}$ which captures all correlations in $\displaystyle\rho$ that cannot be observed in any $\displaystyle k$-party subsystem is the divergence

[TABLE]

from the Gibbs family $\displaystyle\mathcal{E}_{k}$ Weis15 . Here, the divergence $\displaystyle\mathrm{Q}(\rho,\sigma)$ is the quantum relative entropy defined by

[TABLE]

where $\displaystyle{\rm Tr}$ denotes the trace operator on the finite-dimensional Hilbert space of density matrices. Similar to the classical case, we can consider the family $\displaystyle\mathcal{E}_{1}$ of Gibbs states whose closure corresponds to the set of product states $\displaystyle\sigma_{1}\otimes\ldots\otimes\sigma_{n}$. Consider then a composite quantum state $\displaystyle\rho\in\mathcal{A}_{[n]}$ such that

[TABLE]

where $\displaystyle a\in\mathcal{A}_{\{i\}}=\mathcal{A}_{i}$ and $\displaystyle\mathbb{I}_{[n]\backslash\{i\}}$ is the identity operator on the product $\displaystyle\mathcal{A}_{1}\otimes\ldots\hat{\mathcal{A}}_{i}\ldots\otimes\mathcal{A}_{n}$ where $\displaystyle\mathcal{A}_{i}$ is missing. In this case, the many-party correlations of $\displaystyle\rho$ is the quantum multi information:

[TABLE]

where $\displaystyle\widetilde{H}(\rho)=-{\rm Tr}(\rho\log\rho)$ is the von Neumann entropy of $\displaystyle\rho$. In particular, when $\displaystyle n=2$, this corresponds to the quantum mutual information. Algorithms for the evaluation of $\displaystyle\mathrm{Q}(\rho,\mathcal{E}_{k})$ as a complexity measure for quantum states are studied in Niekamp13 . In that context, the many-party correlations is related to the entanglement of quantum systems as defined in Vedral97 .

The scope of the present article is mainly to present the canonical divergence $\displaystyle\mathcal{D}$ defined in Equation (7) as an important tool for generalizing the concept of complexity measure claimed by Equation (10) for classical systems as well as the concept of many-party correlation given by Equation (11) for quantum systems. To this end, we consider the space of density matrices endowed with the quantum analog of the Fisher metric and the mixture $\displaystyle(m)$ and exponential $\displaystyle(e)$ affine connections. This structure turns out to be induced on the manifold of positive density operators by the Bogoliubov inner product Nagaoka95 . In this setting, we prove that the divergence introduced in Ay15 reduces to the quantum relative entropy. In addition, we also show that $\displaystyle\mathcal{D}(\sigma,\rho)=\mathrm{Q}(\sigma,\rho)=\mathcal{D}^{*}(\rho,\sigma)$.

The layout of the paper is as follows. Section II is devoted to the calculation of the canonical divergence and its dual function on the simplex of probability distributions. In Section III, we describe the differential geometrical framework for finite quantum systems induced by the Bogoliubov inner product. In this particular framework, we then prove that the divergence given by Equation (7) reduces to the quantum relative entropy. Finally, we draw some conclusions in Section IV by outlining the results obtained in this work and discussing possible extensions.

II Canonical Divergence on the Simplex of Probability Measures

A dualistic structure on the simplex of probability measures was introduced by Amari in terms of the Fisher metric, the mixture $\displaystyle(m)$ and exponential $\displaystyle(e)$ connections Amari . Given a finite set $\displaystyle I=\{1,\ldots,n\}$, we can represent probability measures on the set $\displaystyle I$ as elements of $\displaystyle\mathbb{R}^{n}$. In this representation, the Dirac measures $\displaystyle\delta^{i},\,i=1,\ldots,n$ form the canonical basis of $\displaystyle\mathbb{R}^{n}$. Then, the $\displaystyle(n-1)$-dimensional simplex of probability measure is given by

[TABLE]

In this section, we show that the canonical divergence $\displaystyle\mathcal{D}(p,q)$ coincides with the Kullback–Leibler divergence whenever $\displaystyle p,q\in\mathcal{S}_{n}$. In addition, we prove that, for the dual canonical divergence, the following relation $\displaystyle\mathcal{D}^{*}(p,q)=\mathrm{KL}(q,p)$ holds true. According to Equations (7) and (8), we need the Fisher metric defined on the tangent bundle $\displaystyle{\rm T}\mathcal{S}_{n}$, the mixture $\displaystyle(m)$-geodesic and the exponential $\displaystyle(e)$-geodesic both connecting $\displaystyle p$ with $\displaystyle q$. On the tangent space $\displaystyle{\rm T}_{p}\mathcal{S}_{n}$, the Fisher metric results in

[TABLE]

The dualistic structure $\displaystyle({\rm g},\nabla,\nabla^{*})$ on $\displaystyle\mathcal{S}_{n}$, given by the Fisher metric, the $\displaystyle(m)$-connection $\displaystyle\nabla$ and the $\displaystyle(e)$-connection $\displaystyle\nabla^{*}$, is dually flat, and the $\displaystyle(m)$- and $\displaystyle(e)$-geodesics connecting $\displaystyle p$ with $\displaystyle q$ are Ay17 :

[TABLE]

We are now ready to compute the canonical divergence $\displaystyle\mathcal{D}(p,q)$ for arbitrary $\displaystyle p,q\in\mathcal{S}_{n}$. From Equations (7), (14), and (15), we have that

[TABLE]

where we use $\displaystyle\sum_{i}(q_{i}-p_{i})=0$ because $\displaystyle p,q\in\mathcal{S}_{n}$. Analogously, we can compute the dual canonical divergence $\displaystyle\mathcal{D}^{*}(p,q)$ by means of Equation (8). Therefore, by using Equations (14) and (16), we obtain that

[TABLE]

To develop further the calculation, let us analyze the derivative $\displaystyle\dot{\gamma}_{e}^{i}(t)$. Recall that

[TABLE]

Therefore, by taking the derivative of $\displaystyle\gamma_{e}^{i}(t)$ with respect to $\displaystyle t$, we obtain

[TABLE]

By stepping back to Equation (18), we start by performing an integration by parts:

[TABLE]

where the last term is obtained by noticing that

[TABLE]

Since we know that

[TABLE]

we can observe that $\displaystyle\frac{\dot{\gamma}_{e}^{i}(1)}{\gamma_{e}^{i}(1)}=\left(\log\frac{q_{i}}{p_{i}}-\sum_{j}q_{j}\log\frac{q_{j}}{p_{j}}\right)$. Hence, we obtain from Equation (19)

[TABLE]

because $\displaystyle p,q\in\mathcal{S}_{n}$. This proves that

[TABLE]

III Geometric Structure of a Manifold of Quantum States

We start this section by showing that natural analogs of the Fisher metric and the exponential and mixture connections are defined on a manifold of quantum states Nagaoka95 . To this end, we need to specify an inner product on the space of density operators. Since the divergence $\displaystyle\mathcal{D}$ of Equation (7) is defined on a statistical manifold $\displaystyle(\mathrm{M},{\rm g},\nabla,\nabla^{*})$ with symmetric connections, we choose the Bogoliubov inner product. This is because of a well-known result that claims the $\displaystyle(e)$-connection induced by a generalized covariance is symmetric if and only if such a covariance is the Bogoliubov inner product Amari00 . At the end of this section, we motivate this choice in more detail.

Let $\displaystyle\mathcal{H}$ be a finite-dimensional Hilbert space, $\displaystyle\mathcal{A}=\{A\,|\,A=A^{*}\}$ be the space of all the Hermitian operators on $\displaystyle\mathcal{H}$ and $\displaystyle\mathcal{S}=\{\rho\,|\,\rho=\rho^{*}>0,\,{\rm Tr}\rho=1\}$ be the space of positive density operators on $\displaystyle\mathcal{H}$. Since $\displaystyle\mathcal{S}$ is an open subset of $\displaystyle\mathcal{A}_{1}:=\{A\,|\,A=A^{*},\,{\rm Tr}A=1\}$, then it can be naturally seen as a smooth manifold of dimension $\displaystyle n=\left(\dim\mathcal{H}\right)^{2}-1$ Nagaoka95 . Let $\displaystyle D\in{\rm T}_{\rho}\mathcal{S}$ be a tangent vector at $\displaystyle\rho$ to $\displaystyle\mathcal{S}$; we call $\displaystyle D^{(m)}\in\mathcal{A}_{0}:=\{A\,|\,A\in\mathcal{A},\,{\rm Tr}A=0\}$ its $\displaystyle(m)$-representation and symbolically write

[TABLE]

It is worth noticing that, as an element of the tangent space, $\displaystyle D$ can be naturally interpreted as a derivative. As an example, when a coordinate system $\displaystyle\{\theta^{i}\}$ is given on $\displaystyle\mathcal{S}$ so that each state is parameterized as $\displaystyle\rho\equiv\rho_{\theta}$, the $\displaystyle(m)$-representation of the natural basis vector is written as $\displaystyle(\partial_{i})^{(m)}=\partial_{i}\rho_{\theta}$, where $\displaystyle D=\partial_{i}=\partial/\partial\theta^{i}$. This allows us to introduce the $\displaystyle(m)$-connection on the manifold $\displaystyle\mathcal{S}$ of the quantum states in terms of the covariant derivative $\displaystyle\nabla^{(m)}:\mathcal{T}(\mathcal{S})\times\mathcal{T}(\mathcal{S})\rightarrow\mathcal{T}(\mathcal{S})$, which is defined by the following relation:

[TABLE]

where the right hand side means the derivative by $\displaystyle X$ of $\displaystyle Y^{(m)}:\mathcal{S}\rightarrow\mathcal{A}_{0}$ and $\displaystyle\mathcal{T}(\mathcal{S})$ denotes the space of sections on $\displaystyle\mathcal{S}$.

To introduce the $\displaystyle(e)$-connection on $\displaystyle\mathcal{S}$, we need to specify a family $\displaystyle\{\langle\cdot,\cdot\rangle_{\rho}\,|\,\rho\in\mathcal{S}\}$ of inner products on $\displaystyle\mathcal{A}$ usually named as generalized covariance. For the reason mentioned above, we consider the Bogoliubov inner product, which is given by

[TABLE]

Given $\displaystyle D\in{\rm T}_{\rho}\mathcal{S}$, we then define the $\displaystyle(e)$-representation of $\displaystyle D$ as the Hermitian operator $\displaystyle D^{(e)}\in\mathcal{A}$ satisfying the following relation:

[TABLE]

For all $\displaystyle A\in\mathcal{A}$, we assume $\displaystyle\langle A,\mathbb{I}\rangle_{\rho}=\langle A\rangle_{\rho}={\rm Tr}(\rho A)$ ($\displaystyle\mathbb{I}$ denotes the identity operator). Thus, we can see that the derivative of the function $\displaystyle\langle A\rangle:\rho\rightarrow\langle A\rangle_{\rho}$ by $\displaystyle D$ is written as

[TABLE]

This implies that we can consider the $\displaystyle(e)$-representation $\displaystyle D^{(e)}\in\mathcal{A}$ of a given $\displaystyle D\in{\rm T}_{\rho}\mathcal{S}$ as

[TABLE]

Therefore, it turns out that $\displaystyle D^{(e)}$ is the derivative of the map $\displaystyle\rho\mapsto\log\rho$ from $\displaystyle\mathcal{S}$ to $\displaystyle\mathcal{A}$, which may be written as follows:

[TABLE]

By considering $\displaystyle\left\langle D^{(e)},\mathbb{I}\right\rangle_{\rho}=\left\langle D^{(e)}\right\rangle_{\rho}={\rm Tr}\left(\rho\,D^{(e)}\right)$, we can immediately observe that

[TABLE]

This proves that, although the $\displaystyle(e)$-representation depends on the choice of the generalized covariance, the space $\displaystyle{\rm T}_{\rho}^{(e)}\mathcal{S}:=\{D^{(e)}\,|\,D\in{\rm T}_{\rho}\mathcal{S}\}$ can be simply written as follows

[TABLE]

This fact supplies the manifold $\displaystyle\mathcal{S}$ of quantum states with the $\displaystyle(e)$-connection. To see this, let us consider the linear isomorphism $\displaystyle D\mapsto D^{\prime}$ from $\displaystyle{\rm T}_{\rho}\mathcal{S}$ to $\displaystyle{\rm T}_{\rho^{\prime}}\mathcal{S}$ defined by $\displaystyle D^{\prime(e)}=D^{(e)}-\left\langle D^{(e)}\right\rangle_{\rho^{\prime}}$. By writing this correspondence as $\displaystyle D^{\prime}=\left[D\right]_{\rho^{\prime}}$, $\displaystyle D=\left[D^{\prime}\right]_{\rho}$, the $\displaystyle(e)$-connection $\displaystyle\nabla^{(e)}$ is then defined by

[TABLE]

where the right hand side means the derivative by $\displaystyle X_{\rho}$ of $\displaystyle\left[Y\right]_{\rho}:\mathcal{S}\rightarrow{\rm T}_{\rho}\mathcal{S}$.

Finally, we define the inner product $\displaystyle{\rm g}_{\rho}$ on $\displaystyle{\rm T}_{\rho}\mathcal{S}$ by

[TABLE]

which is usually called the quantum Fisher metric. The procedure thus far described endows the manifold $\displaystyle\mathcal{S}$ of quantum states with a geometric structure $\displaystyle({\rm g},\nabla^{(e)},\nabla^{(m)})$ given by the quantum Fisher metric, and two torsion-free connections, namely the $\displaystyle(e)$-connection $\displaystyle\nabla^{(e)}$ and the $\displaystyle(m)$-connection $\displaystyle\nabla^{(m)}$, which are dual with respect to $\displaystyle{\rm g}$ in the following sense:

[TABLE]

In addition, the dual structure $\displaystyle({\rm g},\nabla^{(m)},\nabla^{(e)})$ is dually flat, meaning that the curvature tensors of $\displaystyle\nabla^{(e)}$ and $\displaystyle\nabla^{(m)}$ are both null.

Suppose that a coordinate system $\displaystyle\{\xi_{i}\}$ is given and that each element $\displaystyle\rho\in\mathcal{S}$ is specified by the coordinate $\displaystyle\boldsymbol{\xi}\in\mathbb{R}^{n}$ as $\displaystyle\rho\equiv\rho_{\boldsymbol{\xi}}$. According to Equation (20), we have that the mixture representation $\displaystyle\partial_{i}^{(m)}$ of $\displaystyle\partial_{i}=\partial/\partial\xi^{i}$ is given by $\displaystyle\partial^{(m)}_{i}\rho=\partial_{i}\rho_{\boldsymbol{\xi}}$, whereas, by Equation (23), we have that the exponential representation $\displaystyle\partial_{i}^{(e)}$ of $\displaystyle\partial_{i}$ is written as $\displaystyle\partial_{i}^{(e)}\rho=\partial_{i}\log\rho_{\boldsymbol{\xi}}$. Therefore, the dual structure $\displaystyle({\rm g},\nabla^{(e)},\nabla^{(m)})$ with respect to an arbitrary coordinate system $\displaystyle\{\xi^{i}\}$ reads as follows

[TABLE]

A generalized covariance is a family $\displaystyle\{\langle\cdot,\cdot\rangle_{\rho}\,|\,\rho\in\mathcal{S}\}$ of inner products on the space of Hermitian operators $\displaystyle\mathcal{A}$ on the Hilbert space $\displaystyle\mathcal{H}$, where $\displaystyle\langle A,B\rangle_{\rho}$ depends smoothly on $\displaystyle\rho$ for all $\displaystyle A,B\in\mathcal{A}$ and that satisfies the following properties:

• •

For every $\displaystyle U$ unitary matrix on the Hilbert space $\displaystyle\mathcal{H}$, it is

[TABLE]

• •

If the Lie bracket $\displaystyle[\rho,A]=0$, then

[TABLE]

This can be viewed as a quantum version of the $\displaystyle L^{2}$-product

[TABLE]

of random variables $\displaystyle A$ and $\displaystyle B$ with respect to a probability measure $\displaystyle p$. Since $\displaystyle\mathbb{E}_{p}[A,B]$ is the covariance of $\displaystyle A$ and $\displaystyle B$ when their expectations vanish, we can call the family $\displaystyle\{\langle\cdot,\cdot\rangle_{\rho}\,|\,\rho\in\mathcal{S}\}$ satisfying the above conditions a generalized covariance.

According to the theory by Eguchi, a divergence function $\displaystyle\mathcal{D}:\mathrm{M}\times\mathrm{M}\rightarrow\mathbb{R}^{*}$ induces a dual structure $\displaystyle({\rm g},\nabla,\nabla^{*})$ on $\displaystyle\mathrm{M}$ in the way expressed by Equations (4) and (5). It turns out that the connections $\displaystyle\nabla$ and $\displaystyle\nabla^{*}$ obtained in such a way are torsion-free (or symmetric) Eguchi92 . To use the canonical divergence in Equation (7) in the quantum setting, we are then forced to select the Bogoliubov inner product for providing the quantum analog of the Fisher metric, the $\displaystyle(m)$-connection and $\displaystyle(e)$-connection on the manifold of positive density operators. Indeed, while the $\displaystyle(m)$-connection is always torsion-free, it turns out that the $\displaystyle(e)$-connection induced on $\displaystyle\mathcal{S}$ from a generalized covariance is symmetric if and only if such a covariance is the Bogoliubov inner product.

Canonical Divergence on the Manifold of Quantum States

In this section we show that the divergence function of Equation (7) reduces to the quantum relative entropy whenever the dual structure $\displaystyle({\rm g},\nabla^{(m)},\nabla^{(e)})$ on $\displaystyle\mathcal{S}$ is given by the Fisher metric (Equation (28)), the mixture connection (Equation (21)) and the exponential connection (Equation (27)).

Let $\displaystyle\rho_{1},\rho_{2}\in\mathcal{S}$ be two density matrices. To implement the computation of the divergence $\displaystyle\mathrm{D}(\rho_{1},\rho_{2})$ for quantum states, we consider the $\displaystyle(m)$-geodesic $\displaystyle\gamma_{m}(t)=(1-t)\,\rho_{1}+t\,\rho_{2}$ Petz08 . Then, the $\displaystyle(m)$ and $\displaystyle(e)$ representations of the tangent vector $\displaystyle\dot{\gamma}_{m}(t)$ are easily computed by means of Equations (20) and (25), respectively:

[TABLE]

From Equations (7) and (28), we have then

[TABLE]

Let us recall that $\displaystyle\gamma_{m}(t)$ is a curve in the space of density matrices and the logarithm of a positive matrix is a well-defined matrix. Therefore, the derivative with respect to $\displaystyle t$ of $\displaystyle\log\gamma_{m}(t)$ is viewed as the matrix of the derivatives of the entries of $\displaystyle\log\gamma_{m}(t)$ with respect to $\displaystyle t$. Moreover, the same holds for the integration of a matrix: this is the matrix of the integration of the entries. Finally, since the trace is a linear operator it commutes with the integration. Hence, with the abuse of notation where we keep $\displaystyle\gamma_{m}$ instead of the entry $\displaystyle(\gamma_{m})_{ij}$, the computation in Equation (33) is performed as follows by integration by parts:

[TABLE]

This proves that $\displaystyle\mathcal{D}(\rho_{1},\rho_{2})={\rm Tr}\left(\rho_{1}(\log\rho_{1}-\log\rho_{2})\right)$, which is the quantum relative entropy given by Equation (12).

The dual divergence of $\displaystyle\mathcal{D}(\rho_{1},\rho_{2})$ is computed by considering the $\displaystyle(e)$-geodesic connecting $\displaystyle\rho_{1}$ and $\displaystyle\rho_{2}$. Let $\displaystyle\rho_{1}=e^{H}$, where $\displaystyle H$ is a self-adjoint Hamiltonian. Then, the $\displaystyle(e)$-geodesic from $\displaystyle\rho_{1}$ to $\displaystyle\rho_{2}$ is given by

[TABLE]

where $\displaystyle A=\log\rho_{2}-\log\rho_{1}$ and $\displaystyle e^{H+t\,A}$ denotes the exponential matrix Petz08 . Since the trace operator is linear in its argument, it commutes with the derivative operator. Therefore, according to Equations (20) and (25), we obtain that the $\displaystyle(m)$ and $\displaystyle(e)$ representations of $\displaystyle\dot{\gamma}_{e}(t)$ are given by

[TABLE]

The dual divergence of $\displaystyle\mathcal{D}(\rho_{1},\rho_{2})$ is written as follows:

[TABLE]

To perform the computation in Equation (37), we use the expressions of $\displaystyle\dot{\gamma}_{e}^{(m)}$ and $\displaystyle\dot{\gamma}_{e}^{(e)}$ given by Equations (35) and (36):

[TABLE]

At this point, we can use the linearity of the trace operator and then the latter expression reduces to:

[TABLE]

Carrying the integration by parts out, we obtain

[TABLE]

where we use $\displaystyle{\rm Tr}\rho_{1}={\rm Tr}\rho_{2}=1$. This proves that

[TABLE]

IV Conclusions

As we have demonstrated, for a geometric definition of a general complexity measure, it is important to have a canonical divergence. This paper is based on recent progresses in defining a general canonical divergence within Information Geometry Ay15 , Felice18 . This divergence is defined in terms of geodesic integration of the inverse exponential map and holds the geodesic projection property when the structure $\displaystyle({\rm g},\nabla,\nabla^{*})$ is dually flat Ay17 . Let $\displaystyle p\in\mathrm{M}$ and $\displaystyle\widetilde{\mathrm{M}}\subset\mathrm{M}$ be a submanifold of $\displaystyle\mathrm{M}$, the search for $\displaystyle\hat{p}\in\widetilde{\mathrm{M}}$ that minimizes the divergence $\displaystyle\mathcal{D}(p,q),\,q\in\widetilde{\mathrm{M}}$, supplies the solution for defining an information-geometric complexity measure as in Equation (2). When every minimizer $\displaystyle\hat{p}$ of the divergence $\displaystyle\mathcal{D}$ is given by the geodesic projection of $\displaystyle p$ onto $\displaystyle\widetilde{\mathrm{M}}$, we say that $\displaystyle\mathcal{D}$ holds the geodesic projection property. In this regard, the canonical divergence in Equation (7) would provide a measure of complexity as Equation (2) for a quite wide range of systems. A further step for defining Equation (2) for general systems has been put forward in Felice18 , where a new divergence is introduced that turns out to be a generalization of the canonical divergence in Equation (7). As an example of Equation (2), we have considered the measure of complexity given by Equation (10), which quantifies how much a probability measure on the product configuration set of the finitely many states on a discrete set $\displaystyle\{1,\ldots,n\}$ deviates from a family of exponential probabilities that amounts to the non-complex set of system states, as it is given by non-interacting states Aycomplexity . In this case, the Kullback–Leibler divergence turns out to be suitable for providing the measure of complexity in Equation (2) for classic states on discrete sets Ay11 . To put the theory of Ay Aycomplexity in perspective and propose the canonical divergence in Equation (7) as suitable for supplying the complexity in Equation (2) on general systems, we have then proved that $\displaystyle\mathcal{D}$ coincides with the (KL)-divergence on the simplex of probability measures endowed with the dual structure given by the Fisher metric and the mixture and exponential connections.

The quantum counterpart of the general theory yielding the measure of complexity in Equation (2) does not yet exist. However, a quantum analog of Equation (10) has been established on the manifold of positive density operators Weis15 . Here, the family of non-interacting states is replaced by states that are fully described by their restriction to selected subsystems that turn out to be a family of Gibbs states. Therefore, many-party correlations are quantified in the state of composite quantum system, which cannot be observed in subsystems composed of fewer than a given number of parties. The suitable tool for providing such a quantification is established by the quantum relative entropy. This is because the maximum-entropy principle solves the inverse problem to reconstruct a global state from subsystem states and it also gives a natural scale of many-party correlation in terms of the gap to the maximal entropy value. Hence, the many-party correlation of a quantum state is quantified by the divergence from a family of Gibbs state. The many-party correlation in Equation (11) has been implemented in algorithms Niekamp13 proving to be related to the entanglement of quantum systems as defined in Vedral97 . To consider the canonical divergence in Equation (7) as an efficient tool for extending the general theory leading to Equation (2), we have considered $\displaystyle\mathcal{D}$ on the manifold of positive density operators with the quantum analog of the Fisher metric and $\displaystyle(m)$, $\displaystyle(e)$ connections induced by the Bogoliubov inner product. We have finally proved that the canonical divergence coincides with the quantum relative entropy.