Logarithmic divergences: geometry and interpretation of curvature

TL;DR

This paper introduces the logarithmic $L^{(eta)}$-divergence, linking it to optimal transport and geometric structures, and interprets its curvature in the context of statistical manifolds with constant sectional curvature.

Contribution

It establishes the geometric equivalence of the logarithmic divergence to conformal transformations and affine immersions, revealing its role as a canonical divergence in curved statistical manifolds.

Findings

Logarithmic divergence is equivalent to a conformal transformation of Bregman divergence.

The divergence corresponds to a statistical manifold with constant sectional curvature.

Provides a geometric interpretation of curvature in terms of primal and dual geodesics.

Abstract

We study the logarithmic $L^{(\alpha)}$-divergence which extrapolates the Bregman divergence and corresponds to solutions to novel optimal transport problems. We show that this logarithmic divergence is equivalent to a conformal transformation of the Bregman divergence, and, via an explicit affine immersion, is equivalent to Kurose's geometric divergence. In particular, the $L^{(\alpha)}$-divergence is a canonical divergence of a statistical manifold with constant sectional curvature $-\alpha$. For such a manifold, we give a geometric interpretation of its sectional curvature in terms of how the divergence between a pair of primal and dual geodesics differ from the dually flat case. Further results can be found in our follow-up paper [27] which uncovers a novel relation between optimal transport and information geometry.

Equations

$${\mathbf{L}}_{\phi }^{(\alpha )}[\xi :{\xi }^{\prime }]:=\frac{1}{\alpha }\log (1+\alpha D\phi ({\xi }^{\prime })\cdot (\xi -{\xi }^{\prime }))-(\phi (\xi )-\phi ({\xi }^{\prime })),\xi ,{\xi }^{\prime }\in \Omega ,$$

$${\mathbf{B}}_{\varphi }[\xi :{\xi }^{\prime }]:=(\varphi (\xi )-\varphi ({\xi }^{\prime }))-D\varphi ({\xi }^{\prime })\cdot (\xi -{\xi }^{\prime }),$$

$${\mathbf{D}}_{\varphi ,\kappa }[\xi :{\xi }^{\prime }]:=\kappa (\xi ){\mathbf{B}}_{\varphi }[\xi :{\xi }^{\prime }].$$

$$T({\mathbf{L}}_{\phi }^{(\alpha )})\equiv {\mathbf{D}}_{\varphi ,\kappa }.$$

$${\tilde{g}}_{ij}=T^{\prime }(0)g_{ij},{\tilde{\Gamma }}_{ijk}=T^{\prime }(0){\Gamma }_{ijk},{\tilde{\Gamma }}_{ijk}^{\ast }=T^{\prime }(0){\Gamma }_{ijk}^{\ast }.$$

$${\nabla }_{X}Y={\tilde{\nabla }}_{Y}X+\tau (X)Y+\tau (Y)X$$

$$\frac{1}{\kappa (\xi )}\equiv \lambda \varphi (\xi )+a+\sum _{i=1}^n{b}_i{\xi }^i$$

$$\begin{array}{rl}{g}_{ij}(\xi )& =\kappa (\xi ){\partial }_i{\partial }_j\varphi (\xi ),\\ {\Gamma }_{ij}{}^k(\xi )& =\frac{{\partial }_i\kappa (\xi )}{\kappa (\xi )}{\delta }_j^k+\frac{{\partial }_j\kappa (\xi )}{\kappa (\xi )}{\delta }_i^k.\end{array}$$

$$R_{ijk}{}^{\ell }(\xi )=\kappa (\xi )\left({\partial }_{jk}\frac{1}{\kappa }(\xi ){\delta }_i^{\ell }-{\partial }_{ik}\frac{1}{\kappa }(\xi ){\delta }_j^{\ell }\right).$$

$$\kappa (\xi ){\partial }_{jk}\frac{1}{\kappa }(\xi )=\kappa (\xi )g_{jk}(\xi )=\lambda \kappa (\xi ){\partial }_{jk}\varphi (\xi ),$$

$$T_{f(p)}{\mathbb{R}}^{n+1}=f_{\ast }(T_{p}M)\oplus \mathrm{span}(\mathbf{n}(p))$$

$${\overline{\nabla }}_X{f}_{\ast }Y=f_{\ast }({\nabla }_{X}Y)+g(X,Y)\mathbf{n}.$$

$$⟨{\mathbf{n}}^{\ast }(p),\mathbf{n}(p)⟩=1,⟨{\mathbf{n}}^{\ast }(p),f_{\ast }X⟩=0\forall X\in T_{p}M.$$

$$\rho (p,q):=⟨f(p)-f(q),{\mathbf{n}}^{\ast }(q)⟩,p,q\in M.$$

$$\begin{array}{rl}f(\xi )& =\kappa (\xi )({\xi }^1,{\xi }^2,...,{\xi }^n,1),\\ \mathbf{n}(\xi )& =\alpha f(\xi ),\end{array}$$

$$\rho (\xi ,{\xi }^{\prime })={\mathbf{D}}_{\varphi ,\kappa }(\xi ,{\xi }^{\prime })=\frac{1}{\alpha }\left(e^{{\mathbf{L}}_{\phi }^{(\alpha )}[\xi :{\xi }^{\prime }]}-1\right).$$

$${\partial }_k{\tilde{\mathbf{e}}}_j\equiv {\Gamma }_{kj}{}^m{\tilde{\mathbf{e}}}_m+g_{ij}(\alpha f).$$

$${\mathbf{n}}^{\ast }(p_{\xi })=(-{\partial }_1\varphi (\xi ),\dots ,-{\partial }_n\varphi (\xi ),-\alpha \varphi (\xi )+D\varphi (\xi )\cdot \xi ).$$

$$\sec (\mathbf{v},\mathbf{w}):=\frac{⟨R(\mathbf{w},\mathbf{v})\mathbf{v},\mathbf{w}⟩}{\Vert \mathbf{v}{\Vert }^2\Vert \mathbf{w}{\Vert }^2-⟨\mathbf{v},\mathbf{w}{⟩}^2},$$

$$d(x,y):=\underset{\gamma :\gamma (0)=x,\gamma (1)=y}{\inf }{\int }_0^1\Vert \dot{\gamma }(t)\Vert dt,$$

$$\begin{array}{rl}& d^2(r(t_1),p(t_2))\\ & =\Vert \mathbf{v}{\Vert }^2{t}_1^2+\Vert \mathbf{w}{\Vert }^2{t}_2^2-2⟨\mathbf{v},\mathbf{w}⟩t_1{t}_2-\frac{1}{3}⟨R(\mathbf{w},\mathbf{v})\mathbf{v},\mathbf{w}⟩t_1^2{t}_2^2+\cdots ,\end{array}$$

$$\begin{array}{rl}& d^2(r(t_1),p(t_2))-d^2(r(t_1),q)-d^2(q,p(t_2))\\ & =-2⟨\mathbf{v},\mathbf{w}⟩t_1{t}_2-\frac{1}{3}(\Vert \mathbf{v}{\Vert }^2\Vert \mathbf{w}{\Vert }^2-⟨\mathbf{v},\mathbf{w}{⟩}^2)\sec (\mathbf{v},\mathbf{w})t_1^2{t}_2^2+\cdots .\end{array}$$

$$r(t_1)={\exp }_q(t_1\mathbf{v})\text{ and }p(t_2)={\exp }_q^{\ast }(t_2\mathbf{w}),$$

$$H(t_1,t_2):=\mathbf{D}[r(t_1):p(t_2)]-\mathbf{D}[r(t_1):q]-\mathbf{D}[q:p(t_2)].$$

$$\eta =\frac{D\phi (\xi )}{1-\alpha D\phi (\xi )\cdot \xi },\psi (\eta )=\frac{1}{\alpha }\log (1+\alpha \xi \cdot \eta )-\phi (\xi ),$$

$$\mathbf{D}[y:x]=\frac{1}{\alpha }\log (1+\alpha {\xi }_y\cdot {\eta }_x)-\phi ({\xi }_y)-\psi ({\eta }_x).$$

$$\begin{array}{rl}H(t_1,t_2)& =-⟨\mathbf{v},\mathbf{w}⟩t_1{t}_2-\alpha ⟨\mathbf{v},\mathbf{w}⟩\left[\frac{\Vert \mathbf{v}{\Vert }^2}{3}{t}_1^3{t}_2+\frac{\Vert \mathbf{w}{\Vert }^2}{3}{t}_1{t}_2^3+\frac{⟨\mathbf{v},\mathbf{w}⟩}{2}{t}_1^2{t}_2^2\right]\\ & +\mathrm{higher}\mathrm{order}\mathrm{terms}.\end{array}$$

$$⟨\mathbf{v},\mathbf{w}⟩=\left(\frac{v\cdot w}{1+\alpha ({\xi }_q\cdot {\eta }_q)}-\frac{\alpha }{(1+\alpha ({\xi }_q\cdot {\eta }_q){)}^2}({\eta }_q\cdot v)({\xi }_q\cdot w)\right).$$

$$\begin{array}{rl}H(t_1,t_2)=& -⟨\mathbf{v},\mathbf{w}⟩s_1{s}_2+\frac{\alpha ⟨\mathbf{v},\mathbf{w}⟩}{1+\alpha ({\xi }_q\cdot {\eta }_q)}\cdot \left(({\eta }_q\cdot v)s_1^2{s}_2+({\xi }_q\cdot w)s_1{s}_2^2\right)\\ & +(C_3-\alpha C_1{C}_2){\alpha }^2(C_1^2+C_2^2+C_1{C}_2)-\frac{\alpha }{2}(C_3-\alpha C_1{C}_2{)}^2\\ & +\mathrm{higher}\mathrm{order}\mathrm{terms},\end{array}$$

$$s_1(t_1)=t_1+\alpha (D\phi (q)\cdot v)t_1^2+T_1{t}_1^3+O(t_1^4),$$

Full text

1 Introduction

Let $\Omega\subset\mathbf{R}^{n}$ be an open convex set, $n\geq 2$. For $\alpha>0$ fixed, we say that a function $\varphi:\Omega\rightarrow\mathbf{R}$ is $\alpha$-exponentially concave if $e^{\alpha\varphi}$ is concave on $\Omega$. All functions in this paper are assumed to be smooth. Given such a function $\varphi$, we define its $L^{(\alpha)}$-divergence by

[TABLE]

where $D\varphi$ is the Euclidean gradient and $\cdot$ is the dot product. We always assume the Hessian $D^{2}e^{\alpha\varphi}$ is strictly negative definite on $\Omega$. Then $\mathbf{L}^{(\alpha)}_{\varphi}$ is a divergence on $\Omega$, regarded as a manifold, in the sense of [1, Definition 1.1]. As $\alpha\downarrow 0$, the $L^{(\alpha)}$-divergence (with $\varphi$ fixed) converges to the Bregman divergence defined by

[TABLE]

where $\phi=-\varphi$ is convex with $D^{2}\phi>0$. Thus the family $\{{\bf L}_{\varphi}^{(\alpha^{\prime})}\}_{0<\alpha^{\prime}\leq\alpha}$ of logarithmic divergences extrapolate the Bregman divergence $\mathbf{B}_{\phi}$.

Originally motivated by applications in stochastic portfolio theory [7], the $L^{(1)}$-divergence (and its extension to the $L^{(\alpha)}$-divergence) was introduced by Pal and the first author in [19] [26] and was studied further in [24] [20] [25]. There are two main results proved in these papers. First, the $L^{(\alpha)}$-divergence corresponds to the solution to an optimal transport problem with a logarithmic cost function; this is formulated using the general framework of $c$-divergence, see [20] [25] [27]. Also see [21] [9] [18] for recent results about the optimal transport problem which have independent mathematical interest. Second, the induced statistical manifold $(\mathcal{M},g,\nabla,\nabla^{*})$ (see [1, Section 6.2] for the definition) is dually projectively flat with constant sectional curvature $-\alpha$. In [26] we also defined an $L^{(-\alpha)}$-divergence corresponding to constant positive sectional curvature $\alpha$. For expositional simplicity we only consider the $L^{(\alpha)}$-divergence in this paper and [27], but similar results hold for the $L^{(-\alpha)}$-case as well.

In this paper we develop two geometric aspects of the logarithmic divergence. First, we connect the $L^{(\alpha)}$-divergence with classical topics in information geometry, namely conformal transformation and affine differential geometry. In particular, by using an explicit affine immersion, we show that the $L^{(\alpha)}$-divergence is equivalent to the canonical geometric divergence constructed by Kurose [11]. Second, we provide a geometric interpretation of the sectional curvature for a statistical manifold with constant negative sectional curvature. By analyzing a canonical divergence between a pair of primal and dual geodesics, we show that the sectional curvature can be quantified in terms of the deviation from the generalized Pythagorean relation of a dually flat manifold (see Theorem 4.1 below). This extends the geometric interpretation of sectional curvature in Riemannian geometry. In our follow-up work [27] we proved a more general result (see [27, Theorem 3.13]) that holds for any divergence (though it is not intrinsic in the information geometric sense). This was achieved by a novel relation between information geometry and the pseudo-Riemannian framework of Kim and McCann [10] concerning the Ma-Trudinger-Wang tensor in optimal transport.

2 Conformal divergence and its geometry

We refer the reader to [1] for general background in information geometry. Conformal transformations of divergence have been studied in the literature; see for example [17] [12] [2] [15] and the references therein. An important application is robust clustering [23] [14].

Definition 1

Let $\phi:\Omega\rightarrow\mathbb{R}$ be convex (with $D^{2}\phi>0$) and let $\kappa:\Omega\rightarrow(0,\infty)$. We define the (left-sided) conformal transformation of the Bregman divergence $\mathbf{B}_{\phi}$ by

[TABLE]

To abbreviate we call $\mathbf{D}_{\phi,\kappa}$ a conformal divergence.

Note that a right-sided conformal transformation can be converted to a left-sided one by considering the convex conjugate of $\phi$ (see [1, p.17]).

Our first result is that the $L^{(\alpha)}$-divergence is, up to a monotone transformation, equal to a conformal transformation of a Bregman divergence. This shows that the geometry induced by the $L^{(\alpha)}$-divergence can be studied using results of Bregman divergence and conformal transformation.

Theorem 2.1

Consider an $L^{(\alpha)}$-divergence $\mathbf{L}_{\varphi}^{(\alpha)}$ on $\Omega$ as in (1). Let $\phi=-e^{\alpha\varphi}$ which is convex and let $\kappa=-\frac{1}{\alpha\phi}>0$. Then, with $T(x)=\frac{1}{\alpha}(e^{\alpha x}-1)$, we have

[TABLE]

In particular, the conformal divergence $\mathbf{D}_{\phi,h}$ induces the same dualistic structure $(g,\nabla,\nabla^{*})$ as that of $\mathbf{D}_{\varphi}^{(\alpha)}$.

Proof

The identity (4), once conceived, can be verified by a straightforward computation. The second statement is a consequence of the following lemma which can be proved again by a computation. Note that similar reasonings are used in [25, Lemma 3] and [25, Theorem 17]. ∎

Lemma 1

Let $\tilde{\mathbf{D}}$ and $\mathbf{D}$ be divergences related by a monotone transformation: $\tilde{\mathbf{D}}=T(\mathbf{D})$, where $T:[0,\infty)\rightarrow[0,\infty)$ is strictly increasing with $T(0)=0$. Let $(g,\nabla,\nabla^{*})$ and $(\tilde{g},\tilde{\nabla},\tilde{\nabla}^{*})$ be respectively the dualistic structures induced by $\mathbf{D}$ and $\tilde{\mathbf{D}}$. Then, in any local coordinate system, the coefficients of the dualistic structures are related by

[TABLE]

In particular, we have $\tilde{\Gamma}_{ij}\mathstrut^{k}=\Gamma_{ij}\mathstrut^{k}$ and $\tilde{\Gamma}_{ij}^{*}\mathstrut{}^{k}=\Gamma_{ij}\mathstrut^{k}$, and the primal and dual curvature tensors are the same.

Remark 1

By Lemma 1, we say that two divergences ${\bf D}$ and $\tilde{{\bf D}}$ are equivalent if there exists $T$ (as in Lemma 1 with $T^{\prime}(0)=1$) such that $\tilde{\mathbf{D}}=T(\mathbf{D})$. Clearly this defines an equivalence relation among divergences on a manifold. Theorem 2.1 thus states that the $L^{(\alpha)}$-divergence is equivalent to a conformal divergence.

Theorem 2.1 motivates us to study conformal divergences in general. Recall that two torsion-free affine connections $\nabla$ and $\tilde{\nabla}$ are projectively equivalent if there exists a $1$-form $\tau$ such that

[TABLE]

for any vector fields $X$ and $Y$. For its geometric interpretation see [16, p.17]. In particular, $\nabla$ and $\tilde{\nabla}$ have the same geodesics up to time reparameterizations. By definition, $\nabla$ is projectively flat if it is projectively equivalent to a flat connection. When considering the $L^{(\alpha)}$-divergence or a conformal divergence, we think of $\mathcal{M}$ (equal to $\Omega$ as a set) as a manifold, and $\xi$ is the primal (global) coordinate system with values in the convex set $\Omega$.

Proposition 1

Let $(\mathcal{M},g,\nabla,\nabla^{*})$ be the statistical manifold induced by a conformal divergence $\mathbf{D}_{\phi,\kappa}$.

• (i)

The primal connection $\nabla$ is projectively flat and the primal geodesics are, up to time reparameterization, straightlines in the $\xi$-coordinate system. (In fact, using the language of **[3, Section 8.4]**, $\nabla$ is $(-1)$-conformally flat and $\nabla^{*}$ is $1$-conformally flat.)

• (ii)

$\nabla$* has constant sectional curvature $\lambda\in\mathbb{R}$ with respect to $g$ if and only if*

[TABLE]

for some real constants $a$ and $b^{i}$. In this case, the dual sectional curvature is also constant and is equal to $\lambda$.

Remark 2

Note that if (6) holds then one may absorb the linear terms in the definition of $\phi$. On the other hand, we observe that if $\lambda<0$ then $\lambda\phi$ is concave. Since on $\mathbb{R}^{d}$ there are no non-trivial positive concave functions, from (6) we see that if the sectional curvature is constant and negative, the domain $\Omega$ must be a proper subset of $\mathbb{R}^{d}$.

Proof (of Proposition 1)

Consider the dualistic structure $(g,\nabla,\nabla^{*})$ induced by the conformal divergence. Consider the Euclidean coordinate $\xi$ on $\Omega$. By a direct computation, the coefficients of $g$ and $\nabla$ are given by

[TABLE]

Since $\kappa>0$, the $1$-form $\tau=d\log\kappa$ is well-defined. From (7), we have that $\nabla_{X}Y=\tilde{\nabla}_{Y}X+\tau(X)Y+\tau(Y)X$ we $\tilde{\nabla}$ is the Euclidean flat connection on $\Omega$. Thus $\nabla$ is projectively flat and we have (i). A further computation shows that

[TABLE]

Using (8), we see that $\nabla$ has constant sectional curvature $\lambda\in\mathbb{R}$ with respect to $g$ (see [25, Definition 12]) if and only if

[TABLE]

which is equivalent to (6) after integration. ∎

3 Realization by affine immersion

Consider a statistical manifold $(\mathcal{M},g,\nabla,\nabla^{*})$. In [25, Theorem 18] we proved that if both $\nabla$ and $\nabla^{*}$ are dually projectively flat with constant sectional curvature $-\alpha<0$, then one can define intrinsically a local divergence of $L^{(\alpha)}$-type which induces the given geometric structure. In this result, a key idea is that the primal and dual coordinates are related by an optimal transport map (this leads to the self-dual representation given by (21) below). In fact, by [3, Theorem 8.3], if a statistical manifold has constant sectional curvature, then we automatically have dual projective flatness. Thus the condition about projective flatness is redundant and we may modify the statement as follows:

Theorem 3.1

[25, Theorem 18]** The $L^{(\alpha)}$-divergence is a (local) intrinsic divergence for a statistical manifold with constant negative sectional curvature.

On the other hand, for a (simply connected) statistical manifold with constant sectional curvature, Kurose [11] defined globally a canonical, intrinsic divergence using affine differential geometry and proved that it satisfies a generalized Pythagorean theorem. In this section we show that if $(\Omega,g,\nabla,\nabla^{*})$ is induced by an $L^{(\alpha)}$-divergence ${\bf L}_{\varphi}^{(\alpha)}$, then the geometric divergence is the conformal divergence ${\bf D}_{\phi,\kappa}$ in (4). While these canonical divergences are equivalent, our approach in [20] [25] gives an explicit construction in Kurose’s work, covers the Bregman and $L^{(\alpha)}$-divergences under the same framework, and suggests previously unknown connections with optimal transport maps.

To state the main result we recall some concepts of affine differential geometry; for details see [16] and [13]. Let ${\mathcal{M}}$ be an $n$-dimensional manifold. An affine immersion of $M$ into $\mathbb{R}^{n+1}$ consists of an immersion $f:M\rightarrow\mathbb{R}^{n+1}$ and a transversal vector field ${\bf n}$ with values in $\mathbb{R}^{n+1}$ on $M\cong f(M)$. The last statement means that

[TABLE]

for all $p\in M$. Let $\overline{\nabla}$ be the standard (flat) affine connection on $\mathbb{R}^{n+1}$. Then the covariant derivative decomposes as

[TABLE]

We call $\nabla$ and $g$ the induced connection and bilinear form respectively. If the induced connection and bilinear form are equal to the Riemannian metric and primal connection of a dualistic structure $(g,\nabla,\nabla^{*})$, we say that the affine immersion realizes the given structure. By [13, Theorem 5.3], this is possible when the statistical manifold is simply connected and $1$-conformally flat. This is true in particular when the statistical manifold has constant sectional curvature.

Let $(\mathbb{R}^{n+1})^{*}$ be the dual space of $\mathbb{R}^{n}$, and let $\langle.,.\rangle$ be the dual pairing. Given an affine immersion $(f,{\bf n})$, the conormal vector field ${\bf n}^{*}:M\rightarrow(\mathbb{R}^{n+1})^{*}$ is defined by the conditions

[TABLE]

Definition 2 (Kurose’s geometric divergence)

For an affine immersion $(f,{\bf n})$ with conormal field ${\bf n}^{*}$, the geometric divergence is defined by

[TABLE]

In [11] it was shown that if $(g,\nabla)$ is $1$-conformally flat, then the geometric divergence does not depend on the choice of the immersion and recovers the given dualistic structure. (The dual connection $\nabla^{*}$ is uniquely determined given $g$ and $\nabla$.) Hence, it can be viewed as a canonical divergence (see the next section for more discussion).

The following result connects the $L^{(\alpha)}$-divergence with the geometric divergence. It shows that the geometric divergence, the $L^{(\alpha)}$-divergence and the conformal divergence are all equivalent. In particular, they are all intrinsically defined (at least locally) for the given dualistic structure.

Theorem 3.2

Consider a convex domain $\Omega\subset\mathbb{R}^{n}$ equipped with an $L^{(\alpha)}$-divergence ${\bf L}_{\varphi}^{(\alpha)}$ and its induced geometry $(g,\nabla,\nabla^{*})$. Let $\phi=-e^{\alpha\varphi}$ and $\kappa=-\frac{1}{\alpha\phi}$ as in Theorem 2.1. Consider the affine immersion defined by

[TABLE]

where $\xi$ is the Euclidean coordinate system on $\Omega$. Then this affine immersion realizes $(g,\nabla)$. Moreover, the geometric divergence is given by

[TABLE]

Proof

The choice of our immersion (12) is motivated by the proof of [16, Proposition 2.7]. It is easy to see that $f$ is an immersion and ${\bf n}$ is transversal. Let $\tilde{{\bf e}}_{j}:=\frac{\partial}{\partial\xi^{j}}f$ and $\partial_{k}\tilde{{\bf e}}_{j}:=\frac{\partial}{\partial\xi^{k}}\frac{\partial}{\partial\xi^{j}}f$. Then, it can be verified by a straightforward computation that

[TABLE]

We refer the reader to [25, Section 5] for expressions of the coefficients ${\Gamma_{ij}}^{k}$. Thus the affine immersion $(f,{\bf n})$ realizes the given dualistic structure.

Next we construct the conormal vector field. Using the relations in (10), we can show that the conormal field is given by

[TABLE]

We obtain (13) by plugging (15) into (13).

4 Interpretation of sectional curvature

Consider a statistical manifold $(\mathcal{M},g,\nabla,\nabla^{*})$. Given $q\in\mathcal{M}$ and $\mathbf{v},\mathbf{w}\in T_{q}\mathcal{M}$ which are linearly independent, we can define the primal sectional curvature $\mathrm{sec}(\mathbf{v},\mathbf{w})$ by

[TABLE]

where $\langle.,.\rangle$ is the Riemannian inner product and $R$ is the primal curvature tensor. Similarly, we can define the dual sectional curvature $\mathrm{sec}^{*}$. What are the geometric interpretations of these sectional curvatures? Interestingly, to the best of our knowledge, this natural question has not been satisfactorily answered in the literature.

For motivations, let us consider a Riemannian manifold $(\mathcal{M},g)$. In this case, it is well-known that the sectional curvature (given by (16) using the Levi-Civita connection) can be interpreted in terms of the Riemannian distance, defined by

[TABLE]

between a pair of geodesics. For $t_{1},t_{2}>0$ small, let $r(t_{1})=\exp_{q}(t_{1}\mathbf{v})$ and $p(t_{2})=\exp_{q}(t_{2}\mathbf{w})$ be geodesics starting at $q$, where $\exp_{q}$ is the exponential map. Then, we have

[TABLE]

where the higher order terms are omitted (see [22]). This implies that

[TABLE]

We look for analogous geometric interpretations for a statistical manifold. Given a statistical manifold $(\mathcal{M},g,\nabla,\nabla^{*})$, in order to formulate a statement in the form of (18) or (19), we need to have an intrinsically defined divergence corresponding to the given geometry. This is the problem about constructing a canonical divergence and was studied by several papers including [8] [4] [6] [5].

Using the $L^{(\alpha)}$-divergence which is explicit, intrinsically defined and has special properties, in this section we study the geometric interpretation for a statistical manifold with constant sectional curvature $-\alpha\leq 0$. Let $q\in\mathcal{M}$ and ${\bf v},{\bf w}\in T_{q}\mathcal{M}$. Motivated by the generalized Pythagorean theorem which holds for the Bregman and $L^{(\alpha)}$-divergences, let

[TABLE]

where $\exp_{q}$ and $\exp_{q}^{*}$ are respectively the exponential maps corresponding respectively to the primal and dual connections $\nabla$ and $\nabla^{*}$. With $\mathbf{D}$ being an intrinsic local $L^{(\alpha)}$-divergence (see Theorem 3.1), consider the expression $H$ defined by

[TABLE]

By the generalized Pythagorean theorem proved in [20, Theorem 1.2] and [25, Theorem 16], if $\langle\mathbf{v},\mathbf{w}\rangle=0$ then $H(t_{1},t_{2})\equiv 0$. This motivates the definition of $H$ and the comparison with (19). Note that if $\alpha=0$ then the manifold is dually flat. In this case, there is a canonical divergence $\mathbf{D}$ of Bregman type. With the Bregman divergence and with $H$ defined by (20), we have the identity $H(t_{1},t_{2})\equiv-\langle\mathrm{v},\mathrm{w}\rangle t_{1}t_{2}$.

Now let $\alpha>0$ and let $\mathbf{D}$ be the canonical (local) $L^{(\alpha)}$-divergence. By [25, Theorem 18], there exists a local coordinate system $\xi$ and an $\alpha$-exponentially concave function $\varphi=\varphi(\xi)$ such that $\mathbf{D}[y:x]=\mathbf{D}^{(\alpha)}_{\varphi}[\xi_{y}:\xi_{x}]$. Here $\xi_{x}$ is the primal coordinate of $x\in\mathcal{M}$. Moreover, letting

[TABLE]

be respectively the dual coordinate and $\alpha$-conjugate of $\varphi$, we have $\mathbf{D}[y:x]=\mathbf{D}^{(\alpha)}_{\psi}[\eta_{x}:\eta_{y}]$ and the self-dual representation

[TABLE]

As $\alpha\downarrow 0$, these identities reduce to well-known properties of the Bregman divergence [1, Chapter 1]. By analyzing carefully the primal and dual geodesics as well as the self-dual representation (21), we have the following result.

Theorem 4.1

For $t_{1},t_{2}>0$ small, we have

[TABLE]

Proof

By [25, Corollary 2], the primal/dual geodesics of $L^{(\alpha)}$-divergence are straight lines in the primal/dual coordinate systems, up to time changes. Thus we can write $\xi_{r}(t_{1})=\xi_{q}+s_{1}(t_{1})v$ and $\eta_{p}(t_{2})=\eta_{q}+s_{2}(t_{2})w$, where $v$ and $w$ are the coordinate representations of ${\bf v}$ and ${\bf w}$, and $s_{1}$ and $s_{2}$ are time changes. For notational simplicity we suppress the parameters $t_{1}$ and $t_{2}$. Using [25, (89)], we have

[TABLE]

Differentiating (21) and using (23), we expand $H(t_{1},t_{2})$ in terms of $s_{1}$ and $s_{2}$:

[TABLE]

where $C_{1}=\frac{\eta_{q}\cdot v}{1+\alpha(\xi_{q}\cdot\eta_{q})}s_{1}$, $C_{2}=\frac{\xi_{q}\cdot w}{1+\alpha(\xi_{q}\cdot\eta_{q})}s_{2}$, and $C_{3}=\frac{v\cdot w}{1+\alpha(\xi_{q}\cdot\eta_{q})}s_{1}s_{2}$.

On the other hand, the geodesic equations (see [25, (86)]) give us, after some simplifications, Taylor expansions of $s_{1}$ and $s_{2}$:

[TABLE]

[TABLE]

where $T_{1}=\frac{1}{3}(4(\alpha(D\varphi(q)\cdot v))^{2}+\alpha(v^{\top}D^{2}\varphi(q)v))$ and $T_{2}=\frac{1}{3}(4(\alpha(D\psi(q)\cdot w))^{2}+\alpha(w^{\top}D^{2}\psi(q)w))$. The proof is completed by combining (24), (25) and (26). ∎

This result gives a geometric interpretation of the negative sectional curvature $-\alpha$ in terms of the canonical local $L^{(\alpha)}$-divergence $\mathbf{D}$. Note that if we use another intrinsic divergence (such as the conformal divergence) we will get a different expression in (22). Analogous results can be derived for the $L^{(-\alpha)}$-divergence.

Note that Theorem 4.1 implies that $\left.\frac{\partial^{2}}{\partial t_{1}^{2}\partial t_{2}^{2}}{\bf D}[r(t_{1}):p(t_{2})]\right|_{t_{1}=t_{2}=0}=-2\alpha\langle{\bf v},{\bf w}\rangle^{2}$, so the sectional curvature $-\alpha$ may be interpreted in terms of this fourth order mixed derivative. In [27, Theorem 3.13] we extended this result to any divergence. This is formulated using a novel connection between the information geometry of $c$-divergence (which covers all divergences) and the pseudo-Riemannian framework of Kim and McCann [10]. In particular, for any divergence ${\bf D}$, the mixed derivative $\left.\frac{\partial^{2}}{\partial t_{1}^{2}\partial t_{2}^{2}}{\bf D}[r(t_{1}):p(t_{2})]\right|_{t_{1}=t_{2}=0}$ is equal to $-2$ times an un-normalized cross curvature of the Kim-McCann metric induced by the cost function. The reader is referred to [27] for more details. To conclude this paper, let us remark that for a statistical manifold with non-constant sectional curvature, this cross sectional curvature is not intrinsic as there are different divergences (and hence Kim-McCann metrics) which induce the same dualistic structure. A natural starting point is to analyze the canonical divergence of Ay and Amari constructed in [4]. We leave this as a problem for future research.