Subadditivity Beyond Trees and the Chi-Squared Mutual Information

TL;DR

This paper extends the subadditivity of mutual information to series-parallel graphs with asymmetric channels using Chi-squared mutual information, broadening the understanding of information flow in complex networks.

Contribution

It demonstrates that subadditivity holds for asymmetric channels on series-parallel graphs using Chi-squared mutual information, a novel approach compared to previous symmetric channel results.

Findings

Subadditivity applies to asymmetric channels on series-parallel graphs.

Chi-squared mutual information is effective for proving subadditivity in this context.

The new bounds can be tighter than existing SDPI-based bounds.

Abstract

In 2000, Evans et al. [Eva+00] proved the subadditivity of the mutual information in the broadcasting on tree model with binary vertex labels and symmetric channels. They raised the question of whether such subadditivity extends to loopy graphs in some appropriate way. We recently proposed such an extension that applies to general graphs and binary vertex labels [AB18], using synchronization models and relying on percolation bounds. This extension requires however the edge channels to be symmetric on the product of the adjacent spins. A more general version of such a percolation bound that applies to asymmetric channels is also obtained in [PW18], relying on the SDPI, but the subadditivity property does not follow with such generalizations. In this note, we provide a new result showing that the subadditivity property still holds for arbitrary (asymmetric) channels acting on the product of spins, when the graphs are restricted to be series-parallel. The proof relies on the use of the Chi-squared mutual information rather than the classical mutual information, and various properties of the former are discussed. We also present a generalization of the broadcasting on tree model (the synchronization on tree) where the bound from [PW18] relying on the SPDI can be significantly looser than the bound resulting from the Chi-squared subadditivity property presented here.

Equations

$$P_{X,Y}(x,y)=\left(\prod _{(i,j)\in E(G)}{P}_{Y_{ij}|X_i\cdot X_j}(y_{ij}|x_i\cdot x_j)\right)\cdot P_X(x).$$

$$I_2(X_u;X_v\mid Y_{E(G)})\le \sum _{P\in {\mathcal{P}}_G(u,v)}{I}_2(X_u;X_v\mid Y_{E(P)}).$$

$$I_2(X_u;X_W\mid Y)\le {\mathrm{conn}}_{G,\gamma }(u,W),$$

$$\gamma ((i,j))=I_2(X_i;X_j\mid Y_{ij})$$

$$I_2(X_u;X_W\mid Y_{E(G)})$$

$$I_{KL}(X_u;X_W|Y)\le {\mathrm{conn}}_{G,\gamma }(u,W),$$

$$\gamma ((i,j))=\eta (Q_{ij})$$

$${\sigma }_v={\sigma }_{\rho }\prod _e{\eta }_e,$$

$$I_{KL}({\sigma }_{\rho };{\sigma }_W)\le \sum _{w\in W}{I}_{KL}({\sigma }_{\rho };{\sigma }_w).$$

$$I({\sigma }_{\rho };{\sigma }_W)=I(X_{\rho };X_W,Y_{E(T)}),$$

$$I_2(X_{\rho };X_v,Y_{E(G_W)})$$

$$\le \sum _{P\in {\mathcal{P}}_{G_W}(\rho ,v)}{I}_2(X_{\rho };X_v|Y_{E(P)})$$

$$=\sum _{w\in W}\sum _{P\in {\mathcal{P}}_T(\rho ,w)}{I}_2(X_{\rho };X_v|Y_{E(P)\cup (w,v)})$$

$$=\sum _{w\in W}\sum _{P\in {\mathcal{P}}_T(\rho ,w)}{I}_2(X_{\rho };X_w|Y_{E(P)\cup (w,v)})$$

$$=\sum _{w\in W}\sum _{P\in {\mathcal{P}}_T(\rho ,w)}{I}_2(X_{\rho };X_w|Y_{E(P)})$$

$$=\sum _{w\in W}\sum _{P\in {\mathcal{P}}_T(\rho ,w)}{I}_2(X_{\rho };X_w|Y_{E(T)})$$

$$=\sum _{w\in W}\sum _{P\in {\mathcal{P}}_T(\rho ,w)}{I}_2(X_{\rho };X_w,Y_{E(T)})$$

$$=\sum _{w\in W}{I}_2({\sigma }_{\rho };{\sigma }_w)$$

$$I_2(X_{\rho };X_v,Y_{E(G_W)})=I_2(X_{\rho };X_W,Y_{E(T)},Y_{\{(w,v){\}}_{w\in W}})=I_2(X_{\rho };X_W,Y_{E(T)})=I_2({\sigma }_{\rho };{\sigma }_W).$$

$$I_2(X_i;X_j|Y_{ij})=\frac{(a-b{)}^2}{2(a+b)n}+\frac{(a-b{)}^2}{n^2(1-(a+b)/(2n))}=\frac{(a-b{)}^2}{2(a+b)n}+o(1/n)$$

$$I_2(X_u;X_v|Y_{E(G)})$$

$$=\sum _{P\in {\mathcal{P}}_G(u,v)}{\left(\frac{(a-b{)}^2+o(1)}{2(a+b)n}\right)}^{|E(P)|}$$

$$=d^t{\left(\frac{(a-b{)}^2}{2(a+b)n}+o(1/n)\right)}^t$$

$$={\left(\frac{(a-b{)}^2}{2(a+b)}+o_n(1)\right)}^t.$$

$$\eta (Q_{ij})=\frac{(\sqrt{a}-\sqrt{b}{)}^2}{n}+o(1/n).$$

$$I_2(X_u;X_v|Y_{E(G)})$$

$${\mathrm{conn}}_{G,\gamma }(u,v)$$

$$=d^t{\gamma }^t-\frac{d^t{\gamma }^t}{2}\cdot \left(\sum _{k=1}^t{\gamma }^k(d-1)\cdot d^{k-1}\right)$$

$$=d^t{\gamma }^t\left(1-\frac{(d-1)(1-(\gamma d{)}^{t+1})}{2d(1-\gamma d)}\right)$$

$$\ge {\left((\sqrt{a}-\sqrt{b}{)}^2+o_n(1)\right)}^t\left(1-\frac{1}{2}{\left((\sqrt{a}-\sqrt{b}{)}^2+o_n(1)\right)}^{t+1}\right)$$

$$(\sqrt{a}-\sqrt{b}{)}^2\ge \frac{(a-b{)}^2}{2(a+b)},$$

$$I_2(X_u;X_v|Y_{E(G)})$$

$$\le \sum _{P\in {\mathcal{P}}_G(u,v)}{\gamma }^{|E(P)|}$$

$$=d^t\cdot {\gamma }^t$$

$$={\left((\sqrt{a}-\sqrt{b}{)}^2+o_n(1)\right)}^t$$

$$(\sqrt{a}-\sqrt{b}{)}^2>1>\frac{(a-b{)}^2}{2(a+b)},$$

$$I_2(X_u{X}_v;Y_{E(G)})=I_2((X_u{X}_w)\cdot (X_w{X}_v);Y_{E(H_1)},Y_{E(H_2)})$$

$$=I_2(X_u{X}_w;Y_{E(H_1)})I_2(X_w{X}_v;Y_{E(H_2)})$$

$$\le {\sum }_{\begin{array}{c}{P}_1\in {\mathcal{P}}_{H_1}(u,w)\\ P_2\in {\mathcal{P}}_{H_2}(w,v)\end{array}}{I}_2(X_u{X}_w;Y_{E(P_1)})I_2(X_w{X}_v;Y_{E(P_2)})$$

Full text

In this note, we provide a new result showing that the subadditivity property still holds for arbitrary (asymmetric) channels acting on the product of spins, when the graphs are restricted to be series-parallel. The proof relies on the use of the Chi-squared mutual information rather than the classical mutual information, and various properties of the former are discussed.

We also present a generalization of the broadcasting on tree model (the synchronization on tree) where the bound from [PW18] relying on the SPDI can be significantly looser than the bound resulting from the Chi-squared subadditivity property presented here.

1. 1 Introduction

2. 1.1 Spin synchronization model

3. 1.2 Main result

4. 1.3 Comparison to previous results

5. 2 Synchronization on trees: a generalization of broadcasting on trees

6. 2.1 Broadcasting on trees

7. 2.2 Reduction from SOT to BOT

8. 2.3 Asymmetric SOT and Chi-squared v.s. SDPI

9. 3 Proof of Theorem 1.2

10. 4 Future directions

11. A Properties of Chi-squared mutual information

12. B Proof of Theorem 1.3 and Corollary 1.4

1 Introduction

1.1 Spin synchronization model

We consider the problem of reconstructing $n$ independent, uniform $(\pm 1)$-valued spins $X_{1},\dots,X_{n}\stackrel{{\scriptstyle i.i.d.}}{{\sim}}\operatorname{Rad}(1/2)$ living on the vertices of an $n$-vertex graph $G$, by observing their interactions $Y_{e}$ on the edges of the graph. For each $e=(i,j)\in E(G)$, the interaction $Y_{e}=Y_{ij}$ depends only on $X_{i}\cdot X_{j}$. Formally, we can factor the joint distribution as

[TABLE]

In other words, each $Y_{ij}$ is the output of $X_{i}\cdot X_{j}$ through a channel $Q_{ij}:=P_{Y_{ij}|X_{i}\cdot X_{j}}$.

Definition 1.1**.**

We refer to $(X,Y)$ as a spin synchronization model on a graph $G$ with edge channels $Q$.

This spin synchronization model has previously appeared in [Abb+17], for example. Note that in [Abb+17] the alphabets were not restricted to be binary, but were arbitrary groups instead.

Given a spin synchronization problem $(X,Y)$, the goal is to reconstruct the spin $X_{u}$ for some $u\in V(G)$, given $Y$ and $X_{W}$ where $W\subset V(G)$ (due to symmetry, we may as well freeze at least one reference vertex). It is thus natural to look at some measure of dependency between to generic vertices in the graph, and a natural quantity of interest is the mutual information $I(X_{u};X_{W},Y)$.

Connection to other reconstruction models

Depending on the choices of the graph and the edge interaction channels, the spin synchronization model captures models such as (a) broadcasting on trees [Eva+00], (b) censored block models [HLM12, Abb+14], (c) synchronization on grids [Abb+17], and (d) spiked Wigner models [DAM15]. Thus, several information-theoretic reconstruction thresholds for these models can be obtained by studying the information-theoretic thresholds for the spin synchronization model. We refer the reader to [AB18, PW18] for details on these connections.

Connection to Ising Model

The spin synchronization model is also related to the Ising model in statistical physics; conditioned on the edge observations $Y$, the posterior distribution of the vertex spins $X$ is given by an Ising model. However, we will interested here in the average-case behavior over the edge observations in the model, so naively applying correlation decay results on Ising models (e.g., Dobrushin conditions [Dob68]) would yield weaker bounds than those we will obtain.

1.2 Main result

The main contribution of this paper is Theorem 1.2, a new impossibility result for reconstruction in the spin synchronization model.111Part of the results in this note appeared in Enric Boix’s undergraduate senior thesis [Boi18]. This result is incomparable to the previously-known bounds derived in [AB18] and [PW18]. We discuss in detail the relationship between the new bound and the previous bounds, and in Section 2.3 we give examples of spin synchronization problems for which the new bound is tighter. In Section 2.2, we also use Theorem 1.2 to rederive the impossibility result of [Eva+00] for broadcasting on trees.

For our main result, we work with the Chi-squared mutual information, $I_{2}$, instead of the classical KL mutual information, $I_{KL}$. See Appendix A for definitions. As we will see, the Chi-squared mutual information turns out to be a more natural choice for the proof of the theorem. In any case, for our purposes the KL and Chi-squared mutual informations are equal up to a factor of 2, because of the following well-known inequalities $\frac{1}{2}I_{2}(A;U)\leq I_{\mathrm{KL}}(U;A)\leq I_{2}(U;A)$ when $U\sim\operatorname{Rad}(1/2)$, reproved in Appendix A.

We are ready to state our main result.

Theorem 1.2**.**

Let $(X,Y)$ be a spin synchronization model on a series-parallel graph $G$ with terminals $u$ and $v$, and arbitrary edge channels $Q$. Then

[TABLE]

Here $\mathcal{P}_{G}(u,v)$ is the set of paths (self-avoiding walks) from $u$ to $v$ in $G$.

1.3 Comparison to previous results

Comparison to [AB18]

The recent work [AB18] derives a subadditivity bound similar to Theorem 1.2. While the bound of [AB18] applies to spin synchronization models on general graphs, it requires the edge channels of the models to have certain symmetry. In contrast, in Theorem 1.2, we handle general channels, while giving up generality on the base graph. For ease of comparison, we state the result of [AB18] below.

Theorem 1.3** ( [AB18]).**

Let $(X,Y)$ be a spin synchronization model on a graph $G$, such that the edge channels $Q$ are symmetric: for each $e\in E(G)$, there is a measurable transformation $T_{e}=T_{e}^{-1}$ such that $Q_{e}(\cdot|+1)=T_{e}\sharp Q_{e}(\cdot|-1)$. ($T_{e}\sharp$ denotes the push-forward operation.)

Then for all $u\in V(G),W\subset V(G)$,

[TABLE]

where

[TABLE]

for all $(i,j)\in E(G)$. Here, $\mathrm{conn}_{G,\gamma}(u,W)$ denotes the probability that $u$ is in the same open component as some $w\in W$ in a bond percolation on $G$ where each edge $e$ is independently open with probability $\gamma(e)$.

This gives in turn a subadditivity property for the Chi-squared mutual information analogous to Theorem 1.2:

Corollary 1.4** (from [AB18]).**

Let $(X,Y)$ be as in Theorem 1.3. Then for all $u\in V(G),W\subset V(G)$, $I_{2}$ is subadditive over paths:

[TABLE]

where $\mathcal{P}_{G}(u,W)$ is the set of paths (i.e., self-avoiding walks) from $u$ to $W$ in $G$.

Notice that when the edge channels are symmetric, the subadditivity bound of Theorem 1.2 may be weaker than the percolation bound of Theorem 1.3. However, Theorem 1.2 also applies to spin synchronization instances with asymmetric channels, while Theorem 1.2 does not.

For completeness, we include a shortened proof of Theorem 1.3 and Corollary 1.4 in Appendix B.

Comparison to [PW18]

An information-percolation bound is also obtained in [PW18] using an argument based on Strong Data-Processing Inequalities (SDPI). The bound of [PW18] applies to general graphs, to general channels, and also to a model with more general vertex labels. In order to compare the bound of [PW18] with Theorem 1.2, we state the result of [PW18] for binary labels:

Theorem 1.5** (Theorem 2 in [PW18] for spins).**

Let $(X,Y)$ be a spin synchronization model on a graph $G$. Then, for all $u\in V(G),W\subset V(G)$,

[TABLE]

where

[TABLE]

is the SDPI constant for the edge channel $Q_{ij}$. Again, $\mathrm{conn}_{G,\gamma}(u,W)$ denotes the probability that $u$ is in the same open component as some $w\in W$ in a bond percolation on $G$ where each edge $e$ is independently open with probability $\gamma(e)$.

When the channels $Q$ are symmetric in the sense of Theorem 1.3, the SDPI constant $\eta(Q_{ij})$ equals the Chi-squared mutual information $I_{2}(X_{i};X_{j}|Y_{ij})$, and therefore Theorem 1.5 almost generalizes Theorem 1.3, besides for a minor multiplicative factor of 2 resulting from the relationship between $I_{KL}$ and $I_{2}$ (see Proposition A.5).

However, when the channels $Q$ are asymmetric, $\eta(Q_{ij})$ can be significantly larger than $I_{2}(X_{i};X_{j}|Y_{ij})$. As a result, our subadditivity bound in Theorem 1.2 can be used to show that the bound of Theorem 1.5 is loose. We demonstrate this explicitly in Section 2.3 below, by analyzing a spin synchronization problem on a tied-tree model.

2 Synchronization on trees: a generalization of broadcasting on trees

In this section, we first show how the classical broadcasting on tree (BOT) problem of [Eva+00] can be expressed as a synchronization problem on a series-parallel graph consisting of a tied-tree with symmetric channels (Section 2.2). Applying our Theorem 1.2, we then obtain an alternative proof of the impossibility result of [Eva+00] for BOT reconstruction.

We then generalize this BOT model (Section 2.3), using more general channels in the synchronization on tree (SOT) model. For this generalized BOT, we compare the information bound given by our Theorem 1.2 based on the Chi-squared subadditivity with the information bound given by Theorem 1.5 from [PW18] based on the SDPI constant.

2.1 Broadcasting on trees

In the BOT model of [Eva+00], a random variable is broadcast from the root down the edges of a tree, with each edge flipping the variable with some probability. The goal is to reconstruct the root variable by observing of the variables of some of its descendants. Formally, each vertex $v\in V(T)$ of a tree $T$ has a binary hidden label $\sigma_{v}$. The hidden labels are assigned by letting the root $\rho$ have spin $\sigma_{\rho}\sim\operatorname{Rad}(1/2)$, and by defining edge labels $\{\eta_{e}\}\stackrel{{\scriptstyle i.i.d.}}{{\sim}}\operatorname{Rad}(\varepsilon)$, and letting

[TABLE]

where the product is over the edges in the path from $\rho$ to $v$.

In [Eva+00], the following information subadditivity inequality is proved, for any finite set of vertices $W\subset V(T)$:

[TABLE]

This suffices to show that if $T$ is infinite and has percolation threshold $p_{c}(T)$, then, when $(1-2\varepsilon)^{2}<p_{c}(T)$, it is impossible to reconstruct $\sigma_{\rho}$ from observations of the leaf variables at infinite depth.

2.2 Reduction from SOT to BOT

The BOT model with noise parameter $\epsilon$ and tree $T$ that generates the random variables $(\sigma,\eta)$ is equivalent to the synchronization on tree (SOT) model with noise parameter $\epsilon$ and tree $T$ that generates the random variables $(X,Y)$, where the vertex labels $\{X_{v}\}$ are i.i.d. $\operatorname{Rad}(1/2)$, and the edge channels $\{Q_{ij}\}$ are binary symmetric channels (i.e., $Y_{ij}=X_{i}\cdot X_{j}\cdot Z_{ij}$, where the $Z_{ij}$ are i.i.d $\operatorname{Rad}(\varepsilon)$). By equivalent, we mean the following.

Proposition 2.1**.**

In the reconstruction setting just described, for all $W\subset V(T)$,

[TABLE]

where $I$ is any $f$-mutual information (such as $I_{KL}$ or $I_{2}$).

Proof.

First, define $\sigma^{\prime}_{\rho}=X_{\rho}$ and $\sigma^{\prime}_{w}=X_{w}\cdot\prod_{e}Y_{e}$ for all $w\in W$, where the product is over the edges on the path from $\rho$ to $w$. By data-processing and because $\sigma^{\prime}_{W\cup\{\rho\}}\stackrel{{\scriptstyle d}}{{=}}\sigma_{W\cup\{\rho\}}$, we have $I(X_{\rho};X_{W},Y)\geq I(\sigma^{\prime}_{\rho};\sigma^{\prime}_{W})=I(\sigma_{\rho};\sigma_{W})$.

Conversely, let $X^{\prime}_{\rho}=\sigma_{\rho}$, $Y^{\prime}_{e}\stackrel{{\scriptstyle i.i.d}}{{\sim}}\operatorname{Rad}(1/2)$, and $X^{\prime}_{w}=\sigma_{w}\cdot\prod_{e}Y^{\prime}_{e}$. By data-processing and equality of distributions, $I(\sigma_{\rho};\sigma_{W})=I(\sigma_{\rho};\sigma_{W},Y^{\prime})\geq I(X^{\prime}_{\rho};X^{\prime}_{W},Y^{\prime})=I(X_{\rho},X_{W},Y)$. ∎

Since we wish to apply Theorem 1.2 in order to bound $I(X_{\rho};X_{W},Y_{E(T)})$, and since we expressed our theorem in terms of series-parallel graphs with two terminals, we first note that can consider an equivalent varient of SOT using a series-parallel graph by tying the leaves to a terminal with noiseless channels. Namely, used $(X_{V(T)},Y_{E(T)})$ to construct a spin synchronization model $(X_{V(G_{W})},Y_{E(G_{W})})$ on a series-parallel graph $G_{W}$, where $W$ are the leaves of the tree $T$, and $G_{W}$ extends the tree by adding a new vertex $v$ adjacent to all $w\in W$ with noiseless edge channels between any $w$ and $v$. We call $G_{W}$ a “tied-tree” because the leaves $W$ of the tree $T$ are “tied” together in $G_{W}$ through their connections to vertex $v$.

It is easy to show via induction on the number of edges that tied-trees are indeed series-parallel graphs whose two terminals are the root of the original tree, and the new added vertex connecting the leaves. So the bound of Theorem 1.2 applied to $G_{W}$ yields

[TABLE]

We conclude by noting that since Y_{\{(w,v)\}_{w\in W}}\mathchoice{\mathrel{\hbox to0.0pt{\displaystyle\perp\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{\textstyle\perp\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{\scriptstyle\perp\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{\scriptscriptstyle\perp\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}X_{\rho}|X_{W},Y_{E(T)},

[TABLE]

Hence, we have re-derived the BOT subadditivity inequality (1) for the Chi-squared mutual information (and also for the KL mutual information with a factor of 2 by Proposition A.5). This in turn gives the impossibility result of [Eva+00].

2.3 Asymmetric SOT and Chi-squared v.s. SDPI

In Section 2.2, we formulated the spin broadcasting on trees problem of [Eva+00] as a spin synchronization problem on a tied-tree graph, with binary symmetric edge channels. If we relax the edge channels to allow for general (possibly asymmetric) channels, we thus obtain a generalization of the broadcasting on trees problem. Notice that the path subadditivity bound of Theorem 1.2 still applies to this generalized model, as the tied-tree is series-parallel.

In particular, suppose we are given a spin synchronization model $(X,Y)$ on a tied-tree $G$ with root $u$ and terminal $v$ similarly to previous section, but we now take the edge channels $Q$ on the underlying tree to be asymmetrical, i.e., $Q_{ij}(\cdot|+1)\sim\operatorname{Ber}(a/n)$ and $Q_{ij}(\cdot|-1)\sim\operatorname{Ber}(b/n)$, and the leaves of the tree are tied together at $v\in V(G)$ with noiseless edge channels. Suppose moreover that the tree is a regular tree with $d=n$ descendants at each generation and depth $t$.

For any edge $(i,j)$, we have

[TABLE]

by direct calculation using Proposition A.2. Hence, by Theorem 1.2, we have the bound

[TABLE]

On the other hand, as derived in [PW17], the SDPI constant for the edge channels is

[TABLE]

So letting $\gamma((i,j))=\eta(Q_{i,j})$ for all $(i,j)\in E(G)$, the bound of [PW18] (Theorem 1.5) gives

[TABLE]

And by inclusion-exclusion,

[TABLE]

Note that for any $a,b>0$,

[TABLE]

(and the inequality is strict when $a\neq b$), so when $(\sqrt{a}-\sqrt{b})^{2}<1$ and $t$ and $n$ are sufficiently large, the bound (2) derived from Theorem 1.2 is tighter than the bound (5) implied by Theorem 1.5 from [PW18].

Further, applying a union bound on the paths in order to upper-bound the connection probability in the percolation of [PW18], we have:

[TABLE]

Therefore, taking now

[TABLE]

and $t$ such that $t=O(n)$ and $t=\omega_{n}(1)$ (e.g., $t=n$), we have that (4) vanishes as $n$ diverges, and our bound implies that $u$ and $v$ cannot be synchronized non-trivially (a.k.a. reconstruction/weak recovery is not solvable). In contrast, (13) blows up as $n$ diverges, i.e., (13) tends to 1 (as one can always cap the upper-bound to 1).

3 Proof of Theorem 1.2

The result is obtained by using (i) a multiplicative property of the Chi-squared mutual information on paths (see Proposition A.6), and (ii) a subadditivity property of the Chi-squared mutual information on depth-1 trees (see Lemma A.7). Interestingly, (i) does not hold for the classical mutual information, $I_{KL}$, making the Chi-squared mutual information, $I_{2}$, a natural choice for this proof:

Proof (of Theorem 1.2).

In the following, we implicitly use $I_{2}(X_{k}X_{l};Y)=I_{2}(X_{k};X_{l}|Y)$, by Proposition A.3.

The proof is by induction on $|E(G)|$. The base case, $|E(G)|=1$, is trivial. For the inductive step, one of two cases holds:

(Case 1) $G$ is the series composition of $H_{1}$ which is series-parallel with terminals $u,w$, and $H_{2}$, which is series-parallel with terminals $w,v$.

[TABLE]

The inequality is by the inductive hypothesis.

(Case 2) $G$ is the parallel composition of $H_{1}$ and $H_{2}$ both series-parallel, with terminals $u,v$. Then,

[TABLE]

The inductive step follows by the inductive hypothesis, since $\mathcal{P}_{G}(u,v)=\mathcal{P}_{H_{1}}(u,v)\sqcup\mathcal{P}_{H_{2}}(u,v)$. ∎

4 Future directions

Connections with correlation decay

As mentioned in the introduction, fixing the edge observations and applying Ising model correlation decay conditions yields bounds that are not as strong as those we proved in this paper, because the techniques in our paper allow us to deal with the average-case edge observations, while fixing the edge observations and applying the Dobrushin conditions requires us to work with the worst-case edge observations. It would nonetheless be interesting to elaborate on this connection.

Information subadditivity for general graphs

A possible extension would be to prove that Theorem 1.2 applies to spin synchronization models on general graphs $G$, that have general asymmetric edge channels.

Conjecture 4.1** (Generalization of Theorem 1.2).**

Let $(X,Y)$ be a spin synchronization model on any graph $G$, and arbitrary edge channels $Q$. Then, for any $u,v\in V(G)$,

[TABLE]

Here $\mathcal{P}_{G}(u,v)$ is the set of paths (self-avoiding walks) from $u$ to $v$ in $G$.

In particular, Conjecture 4.1 would directly imply the two-community stochastic block-model information-theoretic threshold for impossibility of weak recovery, without having to reduce to the broadcasting on trees model.

Another related subadditivity bound that one could hope for is:

Conjecture 4.2**.**

Let $(X,Y)$ be a spin synchronization model on a graph $G$, and arbitrary edge channels $Q$. Then, for any $u\in V(G)$, $W\subset V(G)$,

[TABLE]

where $I$ is either the Chi-squared mutual information, $I_{2}$, or the KL mutual information, $I_{KL}$.

Applied to spin synchronization models on trees, Conjecture 4.2 would (also) directly imply the information subadditivity inequality (1) from [Eva+00] for broadcasting on trees.

Information subadditivity for non-uniform binary spins

The information subadditivity inequality of Theorem 1.2 does not hold when the spin synchronization model is generalized to allow for non-uniform binary spins.

For example, consider a graph $G$ with two nodes $u,v$, and two parallel edges $e,f$ between the nodes. Suppose we have vertex labels $X_{u},X_{v}\stackrel{{\scriptstyle i.i.d}}{{\sim}}\operatorname{Rad}(\delta)$, and edge labels $Y_{e}=X_{u}X_{v}Z_{e}$ and $Y_{f}=X_{u}X_{v}Z_{f}$, where $Z_{e},Z_{f}\stackrel{{\scriptstyle i.i.d}}{{\sim}}\operatorname{Rad}(\epsilon)$. Then, by direct calculation

[TABLE]

and

[TABLE]

When $\delta=0.2$ and $\epsilon=0.2$, one may check that $I_{2}(X_{u};X_{v}|Y_{e},Y_{f})\geq I_{2}(X_{u};X_{v}|Y_{e})+I_{2}(X_{u};X_{v}|Y_{f})$, contradicting subadditivity.

This leaves open the question of whether there exists a natural generalization of the subadditivity theorem that holds when the vertex labels are non-uniform (possibly with additional factors).

Information subadditivity for general alphabets

Another way to generalize the vertex labels of the spin synchronization model is to draw them from larger alphabets than the binary alphabet. For example, as in [Abb+17], one may consider vertex labels $X_{v}$ that are uniformly chosen from a group $\mathcal{G}$, and edge labels $Y_{uv}$ that are noisy observations of the difference of the endpoints, $X_{v}X_{u}^{-1}$.

However, the subadditivity of the Chi-squared information over paths does not extend in a direct way to these larger alphabets. For example, consider a “spoon”-shaped graph $G$ with vertices $u,v,w$, an edge $e$ between $u$ and $v$, and two parallel edges $f_{1},f_{2}$ between $v$ and $w$. Let the vertex labels $X_{u},X_{v},X_{w}$ be i.i.d uniform in $\mathbb{Z}/4\mathbb{Z}$, and let the edge labels be as follows:

[TABLE]

Notice that $I_{2}(X_{u};X_{w}|Y_{e},Y_{f_{1}},Y_{f_{2}})=I_{2}(X_{u};X_{v}|Y_{e})=1$. Also, $I_{2}(X_{u};X_{w}|Y_{e},Y_{f_{1}})=1/2$ and $I_{2}(X_{u};X_{w}|Y_{e},Y_{f_{2}})=0$. Therefore, the subadditivity over paths $I_{2}(X_{u};X_{w}|Y_{e},Y_{f_{1}},Y_{f_{2}})\leq I_{2}(X_{u};X_{w}|Y_{e},Y_{f_{1}})+I_{2}(X_{u};X_{w}|Y_{e},Y_{f_{2}})$ does not hold.

Nevertheless, it is an interesting problem to find a generalization of information subadditivity when the vertex labels are uniform over groups. For example, it is plausible that the scaling of the information-theoretic threshold for weak recovery in the $k$-community SBM could be recovered from such a generalization.

A converse to Theorem 1.3

When the edge channels of the spin synchronization model are symmetric, Theorem 1.3 from [AB18] and Theorem 1.5 from [PW18] are tight on trees, so one cannot open the edges with lower probability in general. Is there a converse to Theorem 1.3: i.e., is the mutual information lower-bounded by the connection probability on some non-trivial percolation? For example, for some bounded-degree graphs?

Appendix A Properties of Chi-squared mutual information

Definition A.1**.**

For two random variables $A,B$ with joint distribution $\nu_{A,B}$, the Chi-squared mutual information $I_{2}(A;B)$ is the $f$-mutual information $I_{f}(A;B):=D_{f}(\nu_{A,B}||\nu_{A}\times\nu_{B})$ for the choice $f(t)=(t-1)^{2}$. The KL mutual information is the $f$-mutual information with the choice $f(t)=t\log_{2}t$. Here, $D_{f}(\mu||\nu):=\int f\left(\frac{d\mu}{d\nu}\right)d\nu$ is the $f$-divergence introduced in [Csi67], which is well-defined and has desirable properties such as nonnegativity and monotonicity when $\mu\ll\nu$, $f$ is convex, $f$ is strictly convex at 1, and $f(1)=0$. In particular, the $f$-mutual informations have a data processing inequality.

Proposition A.2**.**

Let $A,U$ be jointly-distributed random variables, with $U\in\{-1,+1\}$. Then

[TABLE]

In particular, if $U\sim\operatorname{Rad}(1/2)$, then

[TABLE]

Proof.

Letting $\nu_{Z}$ denote the distribution of $Z$, and $\Omega$ denote the sample set of $A$,

[TABLE]

So, since

[TABLE]

$\nu_{A}$-almost everywhere, we have

[TABLE]

When $U\sim\operatorname{Rad}(1/2)$, we have $\mathbb{E}[U]=0$ and $\operatorname{Var}[U]=1$, so $I_{2}(A;U)=\mathbb{E}[\mathbb{E}[U|A]^{2}]$. ∎

Proposition A.3**.**

Let $(X,Y)$ be a spin synchronization model. Then $I_{2}(X_{u}X_{v};Y)=I_{2}(X_{u};X_{v},Y)$.

Proof.

[TABLE]

∎

Proposition A.4**.**

Let $(X,Y)$ be a spin synchronization model. Then $I_{2}(X_{u};X_{W},Y)=I_{2}(X_{u};X_{W}|Y).$

Proof.

[TABLE]

∎

Proposition A.5**.**

Let $A,U$ be joint random variables, $U\sim\operatorname{Rad}(1/2)$. Then

[TABLE]

where $I_{KL}$ is in bits.

Proof.

As shown in [Eva+00], this follows from the inequalities

[TABLE]

∎

Proposition A.6**.**

Let $U,V,W\stackrel{{\scriptstyle i.i.d}}{{\sim}}\operatorname{Rad}(1/2)$. Let $A$ be the output of a channel on $UW$, and let $B$ be the output of a channel on $WV$. Then

[TABLE]

Proof.

[TABLE]

∎

Lemma A.7**.**

Let $U\sim\operatorname{Rad}(1/2)$. Let $A$ and $B$ be the outputs of two independent channels on $U$. Then $I_{2}(U;A,B)\leq I_{2}(U;A)+I_{2}(U;B).$

Proof.

Let $\nu_{A,B,U}$ denote the joint distribution of $A,B,U$. For simplicity, we prove the lemma when $A,B$ are discrete. For any two random variables $C,D$ with joint distribution $\nu_{C,D}$, $I_{2}(C;D)=D_{(1-1/t)}(\nu_{C}\nu_{D}||\nu_{C,D})$, where $D_{(1/1-t)}$ is the $(1-1/t)$-divergence, so

[TABLE]

We claim Terms (14) and (15) are $\leq 0$, which implies the lemma statement.

We rewrite Term (14), using $\frac{\nu_{A,U}(a,u)}{\nu_{A}(a)\nu_{U}(u)}-1=2\nu_{U|A}(u|a)-1=\mathbb{E}[U|A=a]\cdot u$:

[TABLE]

Define

[TABLE]

Note

[TABLE]

by the subadditivity of $f(x,y)=xy/(x+y)$ for $x,y\geq 0$. (In particular, for all $a,b,c,d\geq 0$, $f(a,b)+f(c,d)\leq f(a+c,b+d)$.) So

[TABLE]

for some $0\leq c_{A}\leq 1$ by (16). Thus, $\eqref{eq:term1}=\mathbb{E}[U|A]^{2}(c_{A}-1)\leq 0$, as desired.

Now we bound Term (15).

[TABLE]

For compactness, write $\alpha_{a}=\nu_{A|U}(a|1),\beta_{a}=\nu_{A|U}(a|-1),\gamma_{b}=\nu_{B|U}(b|1),\delta_{b}=\nu_{B|U}(b|-1)$:

[TABLE]

We conclude by using

[TABLE]

so

[TABLE]

Since $\left(-\frac{(\gamma_{b}-\delta_{b})^{2}}{4(\gamma_{b}+\delta_{b})}\right)\leq 0$ for all $b$, this proves that Term $\eqref{eq:term2}\leq 0$.

∎

Appendix B Proof of Theorem 1.3 and Corollary 1.4

For the sake of completeness, we provide here a compact proof of the main result of [AB18].

Proof (of Theorem 1.3).

(Step 1) Reduce to case $|W|=1$: Construct the graph $G^{\prime}$ by adding a new vertex $v$ adjacent to all $w\in W$, letting $X_{v}\sim\operatorname{Rad}(1/2)$ and defining $Y_{vw}=X_{v}X_{w}$ for all $w\in W$. Now $I_{2}(X_{u};X_{v}|Y_{E(G^{\prime})})=I_{2}(X_{u};X_{W}|Y_{E(G^{\prime})})=I_{2}(X_{u};X_{v}|Y_{E(G)})$, $\mathrm{conn}_{G^{\prime},\gamma^{\prime}}(u,v)=\mathrm{conn}_{G,\gamma}(u,W)$, and $(X,Y)$ is a symmetric spin synchronization instance on $G^{\prime}$. Hence, it suffices to prove the bound $I_{2}(X_{u};X_{W}|Y)\leq\mathrm{conn}_{G,\gamma}(u,W)$ in the case $|W|=1$.

(Step 2) Reduce to BSC case: For each edge channel $Q_{e}(\cdot|\pm 1)$, let $T_{e}$ be the symmetry transformation and define $Z_{e}=\{Y_{e},T_{e}(Y_{e})\}$. By the symmetry property of the edge channels, X\mathchoice{\mathrel{\hbox to0.0pt{\displaystyle\perp\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{\textstyle\perp\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{\scriptstyle\perp\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{\scriptscriptstyle\perp\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}Z. So $\mathcal{L}(X,Y|Z)$, the law of $(X,Y)$ conditioned on $Z$, is almost surely the law of a spin synchronization model $(X^{\prime},Y^{\prime})$ on $G$, where each of the channels $Q^{\prime}_{e}$ is binary-valued (either $Y_{e}$ or $T_{e}(Y_{e})$). Explicitly, for $z\in Z_{e}$,

[TABLE]

and by the symmetry property this equals

[TABLE]

So $Q^{\prime}_{e}$ is a binary symmetric channel. Hence, proving Theorem 1.3 when all the edge channels are BSC yields the general bound:

[TABLE]

Here, $\gamma_{Z}((i,j))=I_{2}(X_{i},X_{j}|Y_{ij},Z_{ij}),$ and for the last equality we use the fact that $\gamma((i,j))=I_{2}(X_{i},X_{j}|Y_{ij})=\mathbb{E}_{Z}[\gamma_{Z}((i,j))]$.

(Step 3) Prove BSC case: Now it only remains to prove $I_{2}(X_{u};X_{v}|Y)\leq\mathrm{conn}_{G,\gamma}(u,v)$ when all edge channels $Q_{e}$ are BSC. Let the flip probability of $Q_{e}$ be $\varepsilon(e)$, and define $\delta(e)=(1-2\varepsilon(e))$. We can assume that $\delta(e)\in[0,1]$, because we lose no information by flipping edge labels deterministically. Also, by direct calculation $\gamma(e)=\delta(e)^{2}.$

The proof goes by induction on $|S_{\delta}|$, where

[TABLE]

In the base case, $|S_{\delta}|=0$, so all edge observations are completely noiseless or completely noisy. Hence, $I_{2}(X_{u},X_{v}|Y)=1$ if there is a path $P$ from $u$ to $v$ whose edges are all noiseless. If there is no such path, then $I_{2}(X_{u},X_{v}|Y)=0$. This is exactly the statement $I_{2}(X_{u},X_{v}|Y)=\mathrm{conn}_{G,\gamma}(u,v)$.

For the inductive step, assume the theorem when the BSC channels are given by $\delta^{\prime}:E(G)\to[0,1]$ with $|S_{\delta^{\prime}}|<|S_{\delta}|$. Pick an arbitrary edge $f\in S_{\delta}$. We will now interpolate between the case in which $\delta(f)=0$, and the case in which $\delta(f)=1$, with the other edge channels held fixed. For any $t\in[0,1]$, let $\delta_{t}:E(G)\to[0,1]$ be given by $\delta_{t}(e)=\delta(e)$ for $e\neq f$, and $\delta_{t}(f)=t$. Define corresponding spin synchronization models $(X_{t},Y_{t})$, and also $\gamma_{t}=\delta_{t}^{2}$. Write

[TABLE]

In order to prove that $I(t)\leq C(t)$ for all $t\in[0,1]$, we need the following claim:

Claim B.1**.**

There is non-decreasing $h:[0,1]\to\mathbb{R}$ such that

[TABLE]

Assume the claim is true. Then, $h(1)=1$, and since $h(t)$ is non-decreasing, $h(t)\leq 1$. Hence,

[TABLE]

The inequality of the last line follows because $I(0)\leq C(0)$ and $I(1)\leq C(1)$ by the inductive hypothesis. The equality of the last line follows by the linearity of the connection probability in the parameter $\gamma_{t}(f)=t^{2}$.

It only remains to prove the claim. Write $E^{\prime}=E(G)\setminus f$. Also write $f=(i,j)$, $A_{t}=X_{t,u}\cdot X_{t,v}$, and $B_{t}=X_{t,i}\cdot X_{t,j}$. By Proposition A.2, and because $Y_{t,E^{\prime}}$ is a subset of $Y_{t}$,

[TABLE]

Since the only edge channel to change with $t$ is $Q_{f}$, we can couple $X_{0}=X_{t}$, and $Y_{0,E^{\prime}}=Y_{t,E^{\prime}}$. Hence, it suffices to prove that the function

[TABLE]

is non-decreasing in $t$, since then we can set $h(t)=\sum_{\sigma\in\{-1,+1\}^{E^{\prime}}}h(t;\sigma)\cdot\mathbb{P}[Y_{0,E^{\prime}}=\sigma],$ which will also be non-decreasing in $t$.

Fix $\sigma\in\{-1,+1\}^{E^{\prime}}$ such that $\mathbb{P}[Y_{t,E^{\prime}}=\sigma]>0$, and let $P_{\alpha\beta}=\mathbb{P}[(A_{t},B_{t})=(\alpha,\beta)\mid Y_{t,E^{\prime}}=\sigma].$ Set $a=P_{1,1}$, $b=P_{1,-1}$, $c=P_{-1,1}$, $d=P_{-1,-1}$. Since Y_{t,f}\mathchoice{\mathrel{\hbox to0.0pt{\displaystyle\perp\hss}\mkern 2.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{\textstyle\perp\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{\scriptstyle\perp\hss}\mkern 2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{\scriptscriptstyle\perp\hss}\mkern 2.0mu{\scriptscriptstyle\perp}}}A_{t},Y_{t,E^{\prime}}|B_{t}, one can explicitly calculate

[TABLE]

Plugging this in and simplifying, if $b=d=0$ or $a=c=0$, then $h(t;\sigma)=0$, which is non-decreasing because it is constant. Otherwise we get $h(t;\sigma)=\frac{16(ad-bc)^{2}}{1-t^{2}(a-b+c-d)^{2}},$ which is non-decreasing on $[0,1]$ because $(a-b+c-d)^{2}<(a+b+c+d)^{2}=1$. This proves the claim.

∎

Lemma B.2**.**

Suppose $(X,Y)$ is a spin synchronization model on a path $P$ with endpoints $u$ and $v$. Then

[TABLE]

Proof.

[TABLE]

∎

Proof (of Corollary 1.4).

The corollary follows from Theorem 1.3, the union bound $\mathrm{conn}_{G,\gamma}(u,v)\leq\sum_{P\in\mathcal{P}_{G}(u,v)}\mathrm{conn}_{P,\gamma}(u,v),$ and Lemma B.2. ∎

Acknowledgements

This research was partly supported by NSF CAREER Award CCF-1552131.