Conditioning in tropical probability theory

TL;DR

This paper introduces a natural conditioning operation for tropical diagrams of probability spaces and proves its Lipschitz continuity relative to the asymptotic entropy distance, advancing the mathematical understanding of tropical probability theory.

Contribution

It defines a new conditioning operation in tropical probability spaces and establishes its Lipschitz continuity, providing foundational insights into tropical probability structures.

Findings

Conditioning operation is Lipschitz continuous.

Operates on tropical diagrams of probability spaces.

Enhances mathematical framework of tropical probability theory.

Abstract

We define a natural operation of conditioning of tropical diagrams of probability spaces and show that it is Lipschitz continuous with respect to the asymptotic entropy distance.

Equations

$$k$$

$${\mathrm{Ent}}_{\ast }:\mathrm{Prob}⟨\mathbf{G}⟩\to {\mathbb{R}}^{\mathbf{G}}$$

$${\mathrm{Ent}}_{\ast }(\mathcal{X}\otimes \mathcal{Y})={\mathrm{Ent}}_{\ast }(\mathcal{X})+{\mathrm{Ent}}_{\ast }(\mathcal{Y})$$

$$\mathrm{kd}(\mathcal{K}):={\Vert {\mathrm{Ent}}_{\ast }\mathcal{Z}-{\mathrm{Ent}}_{\ast }\mathcal{X}\Vert }_1+{\Vert {\mathrm{Ent}}_{\ast }\mathcal{Z}-{\mathrm{Ent}}_{\ast }\mathcal{Y}\Vert }_1$$

$$k(\mathcal{X},\mathcal{Y}):=\inf \left\{\mathrm{kd}(\mathcal{K}):\mathcal{K}=(\mathcal{X}\leftarrow \mathcal{Z}\to \mathcal{Y})\right\}$$

$$\operatorname{\mathbf{q}}(\bar{s})(A):=\frac{1}{n}\cdot\big{|}\!\left\{k\;{\bm{:}}\;s_{k}\in A\right\}\!\big{|}$$

$$q:X^n\to (\Delta X,{\tau }_n)$$

$$X^n$$

$$\Delta \mathcal{S}:=\left\{({\pi }_i)\in \prod _i\Delta S_i:({\sigma }_{ij}{)}_{\ast }{\pi }_i={\pi }_j\right\}$$

$$\Delta S_0$$

$$\Delta S_0\ni {\pi }_0$$

$${\pi }_0(s|u):=\frac{{\pi }_0(s)}{{\pi }_0({\sigma }_{0,i}^{-1}(u))}$$

$$k(\mathcal{X},\mathcal{Y})\le {\int }_{U}k(\mathcal{X}|u,\mathcal{Y})dp(u)+2[[\mathbf{G}]]\cdot \mathrm{Ent}U$$

$$\kappa (\mathcal{X},\mathcal{Y}):=\lim \frac{1}{n}k({\mathcal{X}}^n,{\mathcal{Y}}^n)$$

$$8t\cdot {\int }_t^{\infty }\frac{\phi (t)}{t^2}dt\le D_{\phi }\cdot \phi (t)$$

$$\operatorname{\bm{\kappa}}\big{(}\mathcal{X}(n+m),\mathcal{X}(n)\otimes\mathcal{X}(m)\big{)}\leq C\cdot\varphi(n+m)$$

$$\kappa (\stackrel{ˉ}{}\mathcal{X},\stackrel{ˉ}{}\mathcal{Y}):=\underset{n\to \infty }{\lim }\frac{1}{n}k(\mathcal{X}(n),\mathcal{Y}(n))$$

$${\mathrm{Ent}}_{\ast }:\mathrm{Prob}[\mathbf{G}]$$

$$\displaystyle[\big{(}\mathcal{X}(n)\big{)}_{n\in\mathbb{N}_{0}}]$$

$$\frac{1}{n}k({\mathcal{X}}^n,{\mathcal{H}}_n)\le C(|X_0|,[[\mathbf{G}]])\cdot \sqrt{\frac{{\ln }^{3}n}{n}}$$

$$\phi (t):=2C(|X_0|,[[\mathbf{G}]])\cdot t^{3/4}\ge 2C(|X_0|,[[\mathbf{G}]])\cdot t^{1/2}\cdot {\ln }^{3/2}t$$

$$[\cdot |\cdot ]:\mathrm{Prob}⟨\mathbf{G}⟩\to \mathrm{Prob}[\mathbf{G}]$$

$$[\mathcal{X}|U]:={\int }_{u\in U}\overrightarrow{(\mathcal{X}|u)}d{p}_U(u)$$

$$\mathcal{Y}(n)=\underset{u\in U}{⨂}(\mathcal{X}|u{)}^{\lfloor n\cdot p(u)\rfloor }$$

$$[\mathcal{X}|U]\stackrel{\kappa }{=}\overrightarrow{(\mathcal{X}|u)}$$

$${\mathrm{Ent}}_{\ast }(\mathcal{X}|U):={\int }_U{\mathrm{Ent}}_{\ast }(\mathcal{X}|u)d{p}_U(u)$$

$$[(\mathcal{X}\otimes \mathcal{Y})|(U\otimes V)]=[\mathcal{X}|U]+[\mathcal{Y}|V]$$

$$[(\mathcal{X}\otimes$$

$$={\int }_{U\otimes V}(\overrightarrow{\mathcal{X}|u}+\overrightarrow{\mathcal{Y}|v})dp(u)dp(v)={\int }_U\overrightarrow{\mathcal{X}|u}dp(u)+{\int }_V\overrightarrow{\mathcal{Y}|v}dp(v)$$

$$=[\mathcal{X}|U]+[\mathcal{Y}|V]$$

$$[{\mathcal{X}}^n|U^n]=n\cdot [\mathcal{X}|U]$$

$$\operatorname{\bm{\kappa}}\Big{(}[\mathcal{X}|U],\,[\mathcal{Y}|V]\Big{)}\leq(2\cdot[\mkern-5.0mu[\mathbf{G}]\mkern-5.0mu]+1)\cdot\operatorname{\mathbf{k}}\left(\mathcal{X},\mathcal{Y}\right)$$

Full text

\NewEnviron

proposition[1]

\NewEnviron

theorem[1]

\NewEnviron

corollary[1]

\NewEnviron

lemma[1]

\NewEnviron

tlemma[1]

\NewEnviron

alphatheorem[1]

\NewEnviron

alphaconjecture[1]

\NewEnviron

Proof[1][#1]Proof: \BODY $\boxtimes$

\NewEnvironskipenv

Conditioning in tropical probability theory

R. Matveev

 and 

J. W. Portegies

1. Introduction

In  [MP18],  [MP19a] we have initiated the study of tropical probability spaces and their diagrams: In  [MP18] we endowed (commutative) diagrams of probability spaces with the intrinsic entropy distance and in [MP19a] we defined tropical diagrams as points in the asymptotic cone of the metric space. They are represented by certain sequences of diagrams of probability spaces.

We expect that tropical diagrams will be helpful in the study of information optimization problems, and we have indeed applied them to derive a dimension-reduction result for the shape of the entropic cone in [MP19b].

In the present article we introduce the notion of conditioning on a space in a tropical diagram and show that the operation is Lipschitz-continuous with respect to the asymptotic entropy distance.

It is a rather technical result, and we have therefore decided to treat it in this separate article, but it is an important ingredient in the theory, and in particular we need it for the dimension-reduction result mentioned before.

Given a tuple of finite-valued random variables $(\mathsf{X}_{i})_{i=1}^{n}$ and a random variable $\mathsf{Y}$, one may “condition” the collection $(\mathsf{X}_{i})$ on $\mathsf{Y}$. The result of this operation is a family of $n$-tuples of random variables denoted $(\mathsf{X}_{i}|\mathsf{Y})_{i=1}^{n}$ parameterized by those values of $\mathsf{Y}$ that have positive probability. Each tuple of random variable in this family is defined on a separate probability space.

When passing to the tropical setting the situation is different in the sense that when we condition a tropical diagram $[\mathcal{X}]$ on a space $[Y]$, the result is again a tropical diagram $[\mathcal{X}|Y]$ rather than a family. After recalling some preliminaries in Section 2, we describe the operation of conditioning and prove that the result depends in a Lipschitz way on the original diagram in Section 3.

2. Preliminaries

Our main objects of study are commutative diagrams of probability spaces and their tropical counterparts. In this section we recall briefly the main definitions and results.

2.1. Probability spaces and their diagrams

2.1.1. Probability spaces

By a finite probability space we mean a set with a probability measure, that has finite support. A reduction from one probability space to another is an equivalence class of measure-preserving maps. Two maps are equivalent, if they coincide on a set of full measure. We call a point $x$ in a probability space $X=(\underline{X},p)$ an atom if it has positive weight and we write $x\in X$ to mean $x$ is an atom in $X$ (as opposed to $x\in\underline{X}$ for points in the underlying set). For a probability space $X$ we denote by $|X|$ the cardinality of the support of the probability measure.

2.1.2. Indexing categories

To record the combinatorial structure of a commutative diagrams of probability spaces and reductions we use an object that we call an indexing category. By an indexing category we mean a finite category $\mathbf{G}$ such that for any pair of objects $i,j\in\mathbf{G}$ there is at most one morphism between them either way. In addition, we will assume it satisfies one additional property that we will describe after introducing some terminology. For a pair of objects $i,j\in\mathbf{G}$ such that there is a morphism $\gamma_{ij}:i\stackrel{{\scriptstyle}}{{\rightarrow}}j$, object $i$ will be called an ancestor of $j$ and object $j$ will be called a descendant of $i$. The subcategory of all descendants of an object $i\in\mathbf{G}$ is called an ideal generated by $i$ and will be denoted $\left\lceil i\right\rceil$, while we will call the subcategory consisting of all ancestors of $i$ together with all the morphisms in it a co-ideal generated by $i$ and denote it by $\left\lfloor i\right\rfloor$. (The term filter is also used for co-ideal in the literature about lattices)

The additional property that an indexing category has to satisfy is that for any pair of objects $i,j\in\mathbf{G}$ there exists a minimal common ancestor $\hat{\imath}$, that is $\hat{\imath}$ is an ancestor for both $i$ and $j$ and any other ancestor of them both is also an ancestor of $\hat{\imath}$.

An equivalent formulation of the property above is the following: the intersection of the co-ideals generated by two objects $i,j\in\mathbf{G}$ is also a co-ideal generated by some object $\hat{\imath}\in\mathbf{G}$.

Any indexing category $\mathbf{G}$ is necessarily initial, which means that there exists an initial object, that is an object $i_{0}$ such that $\mathbf{G}=\left\lceil i_{0}\right\rceil$.

A fan in a category is a pair of morphisms with the same domain. A fan $(i\stackrel{{\scriptstyle}}{{\leftarrow}}k\stackrel{{\scriptstyle}}{{\rightarrow}}j)$ is called minimal if for any other fan $(i\stackrel{{\scriptstyle}}{{\leftarrow}}l\stackrel{{\scriptstyle}}{{\rightarrow}}j)$ included in a commutative diagram

[TABLE]

the vertical arrow must be an isomorphism.

For any pair of objects $i,j$ in an indexing category $\mathbf{G}$ there exists a unique minimal fan $(i\stackrel{{\scriptstyle}}{{\leftarrow}}\hat{\imath}\stackrel{{\scriptstyle}}{{\rightarrow}}j)$ in $\mathbf{G}$.

2.1.3. Diagrams

We denote by $\operatorname{\mathbf{Prob}}$ the category of finite probability spaces and reductions. For an indexing category $\mathbf{G}=\left\{i;\,\gamma_{ij}\right\}$, a $\mathbf{G}$-diagram is a functor $\mathcal{X}:\mathbf{G}\stackrel{{\scriptstyle}}{{\rightarrow}}\operatorname{\mathbf{Prob}}$. A reduction $f$ from one $\mathbf{G}$-diagram $\mathcal{X}=\left\{X_{i};\,\chi_{ij}\right\}$ to another $\mathcal{Y}=\left\{Y_{i};\,\upsilon_{ij}\right\}$ is a natural transformation between the functors. It amounts to a collection of reductions $f_{i}:X_{i}\stackrel{{\scriptstyle}}{{\rightarrow}}Y_{i}$ such that the big diagram consisting of all spaces $X_{i}$, $Y_{i}$ and all morphisms $\chi_{ij}$, $\upsilon_{ij}$ and $f_{i}$ is commutative. The category of $\mathbf{G}$-diagrams and reductions will be denoted $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle$. The construction of diagrams could be iterated, thus we can consider $\mathbf{H}$-diagrams of $\mathbf{G}$-diagrams and denote the corresponding category $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle\left\langle\mathbf{H}\right\rangle=\operatorname{\mathbf{Prob}}\left\langle\mathbf{G},\mathbf{H}\right\rangle$. Every $\mathbf{H}$-diagram of $\mathbf{G}$-diagrams can also be considered as $\mathbf{G}$-diagram of $\mathbf{H}$-diagrams, thus there is a natural equivalence of categories $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G},\mathbf{H}\right\rangle\cong\operatorname{\mathbf{Prob}}\left\langle\mathbf{H},\mathbf{G}\right\rangle$.

A $\mathbf{G}$-diagram $\mathcal{X}$ will be called minimal if it maps minimal fans in $\mathbf{G}$ to minimal fans in the target category. The subspace of all minimal $\mathbf{G}$-diagrams will be denoted $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle_{\mathsf{m}}$. In [MP18] we have shown that for any fan in $\operatorname{\mathbf{Prob}}$ or in $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle$ its minimization exists and is unique up to isomorphism.

2.1.4. Tensor product

The tensor product of two probability spaces $X=(\underline{X\mkern-5.0mu}\mkern 5.0mu,p)$ and $Y=(\underline{Y\mkern-5.0mu}\mkern 5.0mu,q)$ is their independent product, $X\otimes Y:=(\underline{X\mkern-5.0mu}\mkern 5.0mu\times\underline{Y\mkern-5.0mu}\mkern 5.0mu,p\otimes q)$ . For two $\mathbf{G}$-diagrams $\mathcal{X}=\left\{X_{i};\,\chi_{ij}\right\}$ and $\mathcal{Y}=\left\{Y_{i};\,\upsilon_{ij}\right\}$ we define their tensor product to be $\mathcal{X}\otimes\mathcal{Y}=\left\{X_{i}\otimes\mathcal{Y};\,\chi_{ij}\times\upsilon_{ij}\right\}$.

2.1.5. Constant diagrams

Given an indexing category $\mathbf{G}$ and a probability space we can form a constant diagram $X^{\mathbf{G}}$ that has all spaces equal to $X$ and all reductions equal to the identity isomorphism. Sometimes when such constant diagram is included in a diagram with another $\mathbf{G}$-diagrams (such as, for example, a reduction $\mathcal{X}\stackrel{{\scriptstyle}}{{\rightarrow}}X^{\mathbf{G}}$) we will write simply $X$ in place of $X^{\mathbf{G}}$.

2.1.6. Entropy

Evaluating entropy on every space in a $\mathbf{G}$-diagram we obtain a tuple of non-negative numbers indexed by objects in $\mathbf{G}$, thus entropy gives a map

[TABLE]

where the target space $\mathbb{R}^{\mathbf{G}}$ is a space of real-valued functions on the set of objects in $\mathbf{G}$ endowed with the $\ell^{1}$-norm. Entropy is a homomorphism in that it satisfies

[TABLE]

2.1.7. Entropy distance

Let $\mathbf{G}$ be an indexing category and $\mathcal{K}=(\mathcal{X}\stackrel{{\scriptstyle}}{{\leftarrow}}\mathcal{Z}\stackrel{{\scriptstyle}}{{\rightarrow}}\mathcal{Y})$ be a fan of $\mathbf{G}$-diagrams. We define the entropy distance

[TABLE]

The intrinsic entropy distance between two $\mathbf{G}$-diagrams is defined to be the infimal entropy distance of all fans with terminal diagrams $\mathcal{X}$ and $\mathcal{Y}$

[TABLE]

The intrinsic entropy distance was introduced in [KSŠ12, Vid12] for probability spaces.

In [MP18] it is shown that the infimum is attained, that the optimal fan is minimal, that $\operatorname{\mathbf{k}}$ is a pseudo-distance which vanishes if and only if $\mathcal{X}$ and $\mathcal{Y}$ are isomorphic and that $\operatorname{\mathsf{Ent}}_{*}$ is a 1-Lipschitz linear functional with respect to $\operatorname{\mathbf{k}}$.

2.2. Diagrams of sets, distributions and empirical

reductions

2.2.1. Distributions on sets

For a set $S$ we denote by $\Delta S$ the collection of all finitely-supported probability distributions on $S$. For a pair of distributions $\pi_{1},\pi_{2}\in\Delta S$ we denote by $\left\|\pi_{1}-\pi_{2}\right\|_{1}$ the total variation distance between them.

For a map $f:S\stackrel{{\scriptstyle}}{{\rightarrow}}S^{\prime}$ between two sets we denote by $f_{*}:\Delta S\stackrel{{\scriptstyle}}{{\rightarrow}}\Delta S^{\prime}$ the induced affine map (the map preserving convex combinations).

For $n\in\mathbb{N}$ define the empirical map $\operatorname{\mathbf{q}}:S^{n}\stackrel{{\scriptstyle}}{{\rightarrow}}\Delta S$ by the assignment below. For $\bar{s}=(s_{1},\dots,s_{n})\in S^{n}$ and $A\subset S$ set

[TABLE]

For a finite probability space $X=(S,p)$ the empirical distribution on $\Delta X$ is the push-forward $\tau_{n}:=\operatorname{\mathbf{q}}_{*}p^{\otimes n}$. Thus

[TABLE]

is a reduction of finite probability spaces. The construction of empirical reduction is functorial, that is for a reduction between two probability spaces $f:X\stackrel{{\scriptstyle}}{{\rightarrow}}Y$ the diagram of reductions

[TABLE]

commutes.

2.2.2. Distributions on diagrams of sets

Let $\operatorname{\mathbf{Set}}$ denote the category of sets and surjective maps. For an indexing category $\mathbf{G}$, we denote by $\operatorname{\mathbf{Set}}\left\langle\mathbf{G}\right\rangle$ the category of $\mathbf{G}$-diagrams in $\operatorname{\mathbf{Set}}$. That is, objects in $\operatorname{\mathbf{Set}}\left\langle\mathbf{G}\right\rangle$ are commutative diagrams of sets indexed by $\mathbf{G}$, the spaces in such a diagram are sets and arrows represent surjective maps, subject to commutativity relations.

For a diagram of sets $\mathcal{S}=\left\{S_{i};\sigma_{ij}\right\}$ we define the space of distributions on the diagram $\mathcal{S}$ by

[TABLE]

If $S_{0}$ is the initial set of $\mathcal{S}$, then there is an isomorphism

[TABLE]

Given a $\mathbf{G}$-diagram of sets $\mathcal{S}=\left\{S_{i};\sigma_{ij}\right\}$ and an element $\pi\in\Delta\mathcal{S}$ we can construct a $\mathbf{G}$-diagram of probability spaces $(\mathcal{S},\pi):=\left\{(S_{i},\pi_{i});\sigma_{ij}\right\}$. Note that any diagram $\mathcal{X}$ of probability spaces has this form.

2.3. Conditioning

Consider a $\mathbf{G}$-diagram of probability spaces $\mathcal{X}=(\mathcal{S},\pi)$, where $\mathcal{S}$ is a diagram of sets and $\pi\in\Delta\mathcal{S}$. Let $X_{0}=(S_{0},\pi_{0})$ be the initial space in $\mathcal{X}$ and $U=X_{i}$ be another space in $\mathcal{X}$. Since $S_{0}$ is initial, there is a map $\sigma_{0,i}:S_{0}\stackrel{{\scriptstyle}}{{\rightarrow}}S_{i}$. Fix an atom $u\in U$ and define the conditioned distribution $\pi_{0}(\cdot|u)$ on $S_{0}$ as the distribution supported in $\sigma^{-1}_{0,i}(u)$ and for every $s\in\sigma^{-1}_{0,i}(u)$ defined by

[TABLE]

Let $\pi(\cdot|u)\in\Delta\mathcal{S}$ be the distribution corresponding to $\pi_{0}(\cdot|u)$ under the isomorphism in (2.1). We define the conditioned $\mathbf{G}$-diagram $\mathcal{X}|u:=(\mathcal{S},\pi(\cdot|u))$.

2.3.1. The Slicing Lemma

In [MP18] we prove the so-called Slicing Lemma that allows to estimate the intrinsic entropy distance between two diagrams in terms of distances between conditioned diagrams. Among the corollaries of the Slicing Lemma is the following inequality. {proposition}p:slicing Let $(\mathcal{X}\stackrel{{\scriptstyle}}{{\leftarrow}}\hat{}\mathcal{X}\stackrel{{\scriptstyle}}{{\rightarrow}}U^{\mathbf{G}})\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G},\Lambda_{2}\right\rangle$ be a fan of $\mathbf{G}$-diagrams of probability spaces and $\mathcal{Y}\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle$ be another diagram. Then

[TABLE]

The fan in the assumption of the the proposition above can often be constructed in the following manner. Suppose $\mathcal{X}$ is a $\mathbf{G}$-diagram and $U=X_{\iota}$ is a space in it for some $\iota\in\mathbf{G}$. We can construct a fan $(\mathcal{X}\stackrel{{\scriptstyle f}}{{\leftarrow}}\hat{}\mathcal{X}\stackrel{{\scriptstyle g}}{{\rightarrow}}U^{\mathbf{G}})\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G},\Lambda_{2}\right\rangle$ by assigning $\hat{X}_{i}$ to be the initial space of the (unique) minimal fan in $\mathcal{X}$ with terminal spaces $X_{i}$ and $U$ and $f_{i}$ and $g_{i}$ to be left and right reductions in that fan, for any $i\in\mathbf{G}$.

2.4. Tropical Diagrams

s:tropical-diagrams A detailed discussion of the topics in this section can be found in [MP19a].

The asymptotic entropy distance between two diagrams of the same combinatorial type is defined by

[TABLE]

A tropical $\mathbf{G}$-diagram is an equivalence class of certain sequences of $\mathbf{G}$-diagrams of probability spaces. Below we describe the type of sequences and the equivalence relation.

A function $\varphi:\mathbb{R}\geq 1\stackrel{{\scriptstyle}}{{\rightarrow}}\mathbb{R}\geq 0$ is called an admissible function if $\varphi$ is non-decreasing and there is a constant $D_{\varphi}$ such that for any $t\geq 1$

[TABLE]

An example of an admissible function will be $\varphi(t)=t^{\alpha}$, for $\alpha\in[0,1)$.

A sequence $\bar{}\mathcal{X}=(\mathcal{X}(n):\,n\in\mathbb{N}_{0})$ of diagrams of probability spaces will be called quasi-linear with defect bounded by an admissible function $\varphi$ if it satisfies

[TABLE]

For example for a diagram $\mathcal{X}$, the sequence $\overrightarrow{\mathcal{X}}:=(\mathcal{X}^{n}:\,n\in\mathbb{N}_{0})$ is $\varphi$-quasi-linear for $\varphi\equiv 0$ (and for any admissible $\varphi$). Such sequences are called linear.

The asymptotic entropic distance between two quasi-linear sequences \bar{}\mathcal{X}=\big{(}\mathcal{X}(n):\,n\in\mathbb{N}_{0}\big{)} and \bar{}\mathcal{Y}=\big{(}\mathcal{Y}(n):\,n\in\mathbb{N}_{0}\big{)} is defined to be

[TABLE]

and sequences are called asymptotically equivalent if $\operatorname{\bm{\kappa}}(\bar{}\mathcal{X},\bar{}\mathcal{Y})=0$. An equivalence class of a sequence $\bar{}\mathcal{X}$ will be denoted $[\mathcal{X}]$ and the totality of all the classes $\operatorname{\mathbf{Prob}}[\mathbf{G}]$. The sum of two such equivalence classes is defined to be the equivalence class of the sequence obtained by tensor-multiplying representative sequences of the summands term-wise. In addition there is a doubly transitive action of $\mathbb{R}_{\geq 0}$ on $\operatorname{\mathbf{Prob}}[\mathbf{G}]$. In [MP19a] the following theorem is proven {theorem}p:tropical-summary Let $\mathbf{G}$ be an indexing category. Then

1. (i)

The space $\operatorname{\mathbf{Prob}}[\mathbf{G}]$ does not depend on the choice of a positive admissible function $\varphi$ up to isometry. 2. (ii)

The space $\operatorname{\mathbf{Prob}}[\mathbf{G}]$ is metrically complete. 3. (iii)

The map $\mathcal{X}\mapsto\overrightarrow{\mathcal{X}}$ is a $\operatorname{\bm{\kappa}}$-$\operatorname{\bm{\kappa}}$-isometric embedding. The space of linear sequences, i.e. the image of the map above, is dense in $\operatorname{\mathbf{Prob}}[\mathbf{G}]$. 4. (iv)

There is a distance-preserving homomorphism from $\operatorname{\mathbf{Prob}}[\mathbf{G}]$ into a Banach space $B$, whose image is a closed convex cone in $B$. 5. (v)

The entropy functional

[TABLE]

is a well-defined 1-Lipschitz linear map.

2.5. Asymptotic Equipartition Property for Diagrams

Among all $\mathbf{G}$-diagrams there is a special class of maximally symmetric ones. We call such diagrams homogeneous, see below for the definition. Homogeneous diagrams come very handy in many considerations, because their structure is easier to describe then that of general diagrams. We show below that among tropical diagrams, those that have homogeneous representatives are dense. It means, in particular, that when considering continuous functionals on the space of diagrams, it suffices to only look at homogeneous diagrams.

2.5.1. Homogeneous diagrams

A $\mathbf{G}$-diagram $\mathcal{X}$ is called homogeneous if the automorphism group $\operatorname{\mathrm{Aut}}(\mathcal{X})$ acts transitively on every space in $\mathcal{X}$, by which we mean that the action is transitive on the support of the probability measure. Homogeneous probability spaces are isomorphic to uniform spaces. For more complex indexing categories this simple description is not sufficient.

2.5.2. Tropical Homogeneous Diagrams

The subcategory of all homogeneous $\mathbf{G}$-diagrams will be denoted $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle_{\mathsf{h}}$ and we write $\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle_{\mathsf{h},\mathsf{m}}$ for the category of minimal homogeneous $\mathbf{G}$-diagrams. These spaces are invariant under the tensor product, thus they are metric Abelian monoids and the general “tropicalization” described in [MP19a] can be performed. Passing to the tropical limit we obtain spaces of tropical (minimal) homogeneous diagrams, that we denote by $\operatorname{\mathbf{Prob}}[\mathbf{G}]_{\mathsf{h}}$ and $\operatorname{\mathbf{Prob}}[\mathbf{G}]_{\mathsf{h},\mathsf{m}}$, respectively.

2.5.3. Asymptotic Equipartition Property

In [MP18] the following theorem is proven {theorem}p:aep-complete Suppose $\mathcal{X}\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle$ is a $\mathbf{G}$-diagram of probability spaces for some fixed indexing category $\mathbf{G}$. Then there exists a sequence $\bar{}\mathcal{H}=(\mathcal{H}_{n})_{n=0}^{\infty}$ of homogeneous $\mathbf{G}$-diagrams such that

[TABLE]

where $C(|X_{0}|,[\mkern-5.0mu[\mathbf{G}]\mkern-5.0mu])$ is a constant only depending on $|X_{0}|$ and $[\mkern-5.0mu[\mathbf{G}]\mkern-5.0mu]$.

The approximating sequence of homogeneous diagrams is evidently quasi-linear with the defect bounded by the admissible function

[TABLE]

Thus, Theorem LABEL:p:aep-complete above states that $\operatorname{\mathsf{L}}(\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle)\subset\operatorname{\mathbf{Prob}}[\mathbf{G}]_{\mathsf{h}}$. On the other hand we have shown in [MP19a], that the space of linear sequences $\operatorname{\mathsf{L}}(\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle)$ is dense in $\operatorname{\mathbf{Prob}}[\mathbf{G}]$. Combining the two statements we get the following theorem.

{theorem}

p:aep-tropical For any indexing category $\mathbf{G}$, the space $\operatorname{\mathbf{Prob}}[\mathbf{G}]_{\mathsf{h}}$ is dense in $\operatorname{\mathbf{Prob}}[\mathbf{G}]$. Similarly, the space $\operatorname{\mathbf{Prob}}[\mathbf{G}]_{\mathsf{h},\mathsf{m}}$ is dense in $\operatorname{\mathbf{Prob}}[\mathbf{G}]_{\mathsf{m}}$.

3. Conditioning of Tropical Diagrams

3.1. Motivation

Let $\mathcal{X}\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle$ be a $\mathbf{G}$-diagram of probability spaces containing probability space $U=X_{i_{0}}$ indexed by an object $i_{0}\in\mathbf{G}$.

Given an atom $u\in U$ we can define a conditioned diagram $\mathcal{X}|u$. If the diagram $\mathcal{X}$ is homogeneous, then the isomorphism class of $\mathcal{X}|u$ is independent of $u$, so that $\mathcal{X}|u$ is a constant family. On the other hand we have shown, that the power of any diagram can be approximated by homogeneous diagrams, thus suggesting that in the tropical setting $\mathcal{X}|U$ should be a well-defined tropical diagram, rather than a family. Below we give a definition of tropical conditioning operation and prove its consistency.

3.2. Classical-tropical conditioning

Here we define the operation of conditioning of classical diagram, such that the result is a tropical diagram. Let $\mathcal{X}$ be a $\mathbf{G}$-diagram of probability spaces and $U$ be a space in $\mathcal{X}$. We define the conditioning map

[TABLE]

by conditioning $\mathcal{X}$ by $u\in U$ and averaging the corresponding tropical diagrams:

[TABLE]

where $\overrightarrow{(\mathcal{X}|u)}$ is the tropical diagram represented by a linear sequence generated by $\mathcal{X}|u$, see section 2.4. Note that the integral on the right-hand side is just a finite convex combination of tropical diagrams. Expanding all the definitions we will get for $[\mathcal{Y}]:=[\mathcal{X}|U]$ the representative sequence

[TABLE]

3.3. Properties

3.3.1. Conditioning of Homogeneous Diagrams

If the diagram $\mathcal{X}$ is homogeneous, then for any atom $u\in U$ with positive weight

[TABLE]

3.3.2. Entropy

Recall that earlier we have defined a quantity

[TABLE]

Now that $[\mathcal{X}|U]$ is a tropical diagram, the expression $\operatorname{\mathsf{Ent}}_{*}(\mathcal{X}|U)$ can be interpreted in two, a priori different, ways: by the formula above and as the entropy of the object introduced in the previous subsection. Fortunately, the numeric value of it does not depend on the interpretation, since the entropy is a linear functional on $\operatorname{\mathbf{Prob}}[\mathbf{G}]$.

3.3.3. Additivity

If $\mathcal{X}$ and $\mathcal{Y}$ are two $\mathbf{G}$-diagrams with $U:=X_{\iota}$, $V:=Y_{\iota}$ for some $\iota\in\mathbf{G}$, then

[TABLE]

Proof:

[TABLE]

$\boxtimes$

3.3.4. Homogeneity

s:cond-homo It follows that for any diagram $\mathcal{X}$ with a space $U$ and $n\in\mathbb{N}_{0}$ holds

[TABLE]

3.4. Continuity and Lipschitz property

{proposition}

p:cond-lip Let $\mathbf{G}$ be a complete poset category, $\mathcal{X},\mathcal{Y}\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G}\right\rangle$ be two $\mathbf{G}$ diagrams, $U:=X_{\iota}$ and $V:=Y_{\iota}$ be two spaces in $\mathcal{X}$ and $\mathcal{Y}$, respectively, indexed by some $\iota\in\mathbf{G}$. Then

[TABLE]

Using homogeneity property of conditioning, Section 3.3.4, we can obtain the following stronger inequality. {corollary}p:cond-lip-aikd In the setting of Proposition LABEL:p:cond-lip holds

[TABLE]

Before we prove Proposition LABEL:p:cond-lip we will need some preparatory lemmas.

{lemma}

p:dist-cond-types Let $\mathcal{A}$ be a $\mathbf{G}$-diagram of probability spaces and $E$ be a space in it. Let $\mathbf{q}:E^{n}\stackrel{{\scriptstyle}}{{\rightarrow}}(\Delta E,\tau_{n})$ be the empirical reduction. Then for any $n\in\mathbb{N}$ and any $\bar{e},\bar{e}^{\prime}\in E^{n}$

[TABLE]

Proof: To prove the lemma we construct a coupling between $\mathcal{A}^{n}|\bar{e}$ and $\mathcal{A}^{n}|\bar{e}^{\prime}$ in the following manner. Note that there exists a permutation $\sigma\in S_{n}$ such that

[TABLE]

Let

[TABLE]

Using that $|\tilde{I}|=\frac{n}{2}\cdot\|\mathbf{q}(\bar{e})-\mathbf{q}(\bar{e}^{\prime})\|_{1}$ we can estimate

[TABLE]

where $\mathcal{A}\stackrel{{\scriptstyle=}}{{\leftrightarrow}}\mathcal{B}$ denotes the isomorphism coupling of two naturally isomorphic diagrams, while $\mathcal{A}\stackrel{{\scriptstyle\otimes}}{{\leftrightarrow}}\mathcal{B}$ denotes the “independence” coupling. $\boxtimes$

{lemma}

p:int-dist-cond Let $\mathcal{A}$ be a $\mathbf{G}$-diagram of probability spaces and $E$ be a space in $\mathcal{A}$. Then

[TABLE]

Proof: First we apply Proposition LABEL:p:slicing slicing the first argument

[TABLE]

We will argue now that the double integral on the right-hand side grows sub-linearly with $n$. We estimate the double integral by applying Lemma LABEL:p:dist-cond-types to the integrand

[TABLE]

where the convergence to zero of the last double integral follows from Sanov’s theorem. $\boxtimes$

{corollary}

p:dist-cond Let $\mathcal{A}$ be a $\mathbf{G}$-diagram and $E$ a probability space included in $\mathcal{A}$. Then

[TABLE]

Proof: Let $n\in\mathbb{N}$. Then

[TABLE]

where we used Lemma LABEL:p:int-dist-cond and the fact that $\operatorname{\bm{\kappa}}\leq\operatorname{\mathbf{k}}$ in the last line. We finish the proof by taking the limit $n\stackrel{{\scriptstyle}}{{\rightarrow}}\infty$. $\boxtimes$

Proof(of Proposition LABEL:p:cond-lip):

We start with a note on general terminology: a reduction $f:A\stackrel{{\scriptstyle}}{{\rightarrow}}B$ of probability spaces can also be considered as a fan $\mathcal{F}:=(A\stackrel{{\scriptstyle=}}{{\leftarrow}}A\stackrel{{\scriptstyle f}}{{\rightarrow}}B)$. Then the entropy distance of $f$ is

[TABLE]

If the reduction $f$ is a part of a bigger diagram containing also space $U$, then the following inequality holds

[TABLE]

Let $\mathcal{K}\in\operatorname{\mathbf{Prob}}\left\langle\mathbf{G},\bm{\Lambda}_{2}\right\rangle$

[TABLE]

be an optimal coupling between $\mathcal{X}$ and $\mathcal{Y}$. It can also we viewed as a $\mathbf{G}$-diagram of two-fans, $\mathcal{K}=\left\{\mathcal{K}_{i}\right\}_{i\in\mathbf{G}}$ each of which is a minimal coupling between $X_{i}$ and $Y_{i}$. Among them is the minimal fan $\mathcal{W}:=\mathcal{K}_{\iota}=(U\stackrel{{\scriptstyle f_{\iota}}}{{\longleftarrow}}W\stackrel{{\scriptstyle g_{\iota}}}{{\longrightarrow}}V)$.

We use triangle inequality to bound the distance \operatorname{\bm{\kappa}}\Big{(}[\mathcal{X}|U],[\mathcal{Y}|V]\Big{)} by four summands as follows.

[TABLE]

We will estimate each of the four summands separately. The bound for the first one is as follows.

[TABLE]

An analogous calculation shows that

[TABLE]

To bound the second summand we will use Corollary LABEL:p:dist-cond

[TABLE]

We will now use Corollary LABEL:p:dist-cond with $\mathcal{A}=\mathcal{Z}|u$ and $E=W|u$ to estimate the integrand. Then,

[TABLE]

Similarly

[TABLE]

Combining the estimates we get

[TABLE]

$\boxtimes$

3.5. Tropical conditioning

Let $[\mathcal{X}]$ be a tropical $\mathbf{G}$-diagram and $[U]=[X_{\iota}]$ for some $\iota\in\mathbf{G}$. Choose a representative \big{(}\mathcal{X}(n)\big{)}_{n\in\mathbb{N}_{0}} and denote $u(n):=X_{\iota}(n)$. We define now a conditioned diagram $[\mathcal{X}|U]$ by the following limit

[TABLE]

Proposition LABEL:p:cond-lip-aikd guarantees, that the limit exists and is independent of the choice of representative. For a fixed $\iota\in\mathbf{G}$ the conditioning is a linear Lipschitz map

[TABLE]