Double-Edge Factor Graphs: Definition, Properties, and Examples

TL;DR

This paper introduces double-edge factor graphs (DE-FGs), a new class allowing complex-valued local functions with positive semi-definiteness, and explores their properties and applications, including quantum information processing.

Contribution

It defines DE-FGs, analyzes their sum-product algorithm behavior, and demonstrates promising numerical results with connections to quantum information.

Findings

SPA can be effectively applied to DE-FGs.

DE-FGs accommodate complex-valued functions with positive semi-definiteness.

Numerical experiments show promising results in quantum information contexts.

Abstract

Some of the most interesting quantities associated with a factor graph are its marginals and its partition sum. For factor graphs \emph{without cycles} and moderate message update complexities, the sum-product algorithm (SPA) can be used to efficiently compute these quantities exactly. Moreover, for various classes of factor graphs \emph{with cycles}, the SPA has been successfully applied to efficiently compute good approximations to these quantities. Note that in the case of factor graphs with cycles, the local functions are usually non-negative real-valued functions. In this paper we introduce a class of factor graphs, called double-edge factor graphs (DE-FGs), which allow local functions to be complex-valued and only require them, in some suitable sense, to be positive semi-definite. We discuss various properties of the SPA when running it on DE-FGs and we show promising numerical results for various example DE-FGs, some of which have connections to quantum information processing.

Equations

$$g(\mathbf{x},{\mathbf{x}}^{\prime };\mathbf{y})$$

$$\cdot f_1(x_1,x_2,x_1^{\prime },x_2^{\prime };y_3)\cdot f_2(x_2,x_3,x_5,x_2^{\prime },x_3^{\prime },x_5^{\prime })$$

$$\cdot f_3(x_3,x_4,x_3^{\prime },x_4^{\prime };y_2)\cdot f_4(x_4,x_1,x_4^{\prime },x_1^{\prime };y_1).$$

$$g(\mathbf{x},{\mathbf{x}}^{\prime };\mathbf{y})$$

$$\sum _{{\mathbf{x}}_{\partial f},{\mathbf{x}}_{\partial f}^{\prime }}\overline{h({\mathbf{x}}_{\partial f})}\cdot f({\mathbf{x}}_{\partial f},{\mathbf{x}}_{\partial f}^{\prime };{\mathbf{y}}_{\delta f})\cdot h({\mathbf{x}}_{\partial f}^{\prime })$$

$$f({\mathbf{x}}_{\partial f},{\mathbf{x}}_{\partial f}^{\prime };{\mathbf{y}}_{\delta f})$$

$$Z$$

$$\tilde{M}(x_3,x_4,x_3^{\prime },x_4^{\prime };y)$$

$$Z_{\mathrm{Bethe}}$$

$${\mu }_{f_i\to f_{i+1}}^{(t)}({\tilde{x}}_{i+1})$$

$${\mu }_{f_i\to f_{i-1}}^{(t)}({\tilde{x}}_i)$$

$$Z_{\mathrm{Bethe}}$$

$$Z$$

$$\displaystyle f^{\mathrm{L}}_{i}\bigl{(}\{x^{\mathrm{L}}_{i,j}\}_{j\in[n]}\bigr{)}$$

$$\displaystyle f_{i,j}\bigl{(}x^{\mathrm{L}}_{i,j},x^{\mathrm{R}}_{i,j}\bigr{)}$$

$$\displaystyle\tilde{f}^{\mathrm{L}}_{i}\bigl{(}\{\tilde{x}^{\mathrm{L}}_{i,j}\}_{j\in[n]}\bigr{)}$$

$$\displaystyle\tilde{f}_{i,j}\bigl{(}\tilde{x}^{\mathrm{L}}_{i,j},\tilde{x}^{\mathrm{R}}_{i,j}\bigr{)}$$

Full text

In this paper we introduce a class of factor graphs, called double-edge factor graphs (DE-FGs), which allow local functions to be complex-valued and only require them, in some suitable sense, to be positive semi-definite. We discuss various properties of the SPA when running it on DE-FGs and we show promising numerical results for various example DE-FGs, some of which have connections to quantum information processing.

I Introduction

On the one hand, many classical algorithms like Kalman filtering, the BCJR algorithm, the forward-backward algorithm, etc., can be seen as special cases of the sum-product algorithm (SPA) applied to suitable cycle-free factor graphs [1, 2]. On the other hand, the SPA has also been successfully applied to various classes of factor graphs with cycles, as is for example witnessed by the SPA-based decoding techniques of low-density parity-check (LDPC) codes, which appear nowadays in various telecommunication standards [1, 2].

For the case of SPA on factor graphs with cycles, there are a few results that hold for large classes of factor graphs (like the result by Yedidia et al. [3], which states that fixed points of the SPA correspond to stationary points of the Bethe free energy function) or the graph-cover-based interpretation of the Bethe approximation of the partition sum [4], but in general the results are for special classes of factor graphs like Gaussian graphical models (see, e.g., [5]) or log-supermodular (“attractive”) graphical models (see, e.g., [6]). In most of these cases, the focus has been on factor graphs with non-negative real-valued local functions. However, there are applications, in particular in the area of quantum information processing, where one would like to have more general factor graphs. Let us mention some of the approaches that have been pursued:

• •

One approach replaces scalar-valued local functions by matrix-valued local functions (see, e.g., [7, 8]).

• •

Another approach keeps scalar-valued local functions, but imposes certain symmetry conditions on the factor graph [9, 10]. (See the discussion and the references in [10] on how the factor graphs therein are related to tensor networks, etc.) The framework in [9, 10] can, for example, be conveniently used for estimating information rates of channels with a classical input and output and a quantum memory [11].

Note that all marginal calculations were done exactly in [9, 10, 11]. This can be achieved, for example, by first merging suitable variables so that the resulting factor graph is cycle free and then to apply the SPA. (Of course, this only gives practical algorithms as long as the alphabet sizes of the merged variables are not too large.)

However, similar to the above-mentioned classes of factor graphs with cycles, it is tempting to also apply the SPA to factor graphs as in [9, 10] with cycles. There are different approaches to accomplish this by suitably reformulating the factor graphs in [9, 10], some reformulations having better complexity properties, some reformulations having better analytical properties. An interesting option in this design space are the double-edge factor graphs (DE-FGs) that we introduce in this paper.111As we will see, the name “double-edge” comes from the fact that pairs of edges (and with that the associated variables) are merged. For example, referring to in Fig. 2 (top), the edge associated with variable $x_{0}$ and the edge associated with variable $x^{\prime}_{0}$ are merged to a double-edge in Fig. 2 (bottom).

This paper is structured as follows. We define DE-FGs in Section II and then formulate the SPA, along with some of its properties, in Section III. We discuss a variety of examples in Section IV, we point out connections to a recent paper by Mori in Section V, and we conclude the paper in Section VI. Note that throughout this paper, all alphabets are assumed to be finite.

II Double-edge Factor Graphs

In this section we define double-edge factor graphs (DE-FGs), more precisely, double-edge normal factor graphs (DE-NFGs). The word “normal” refers to the fact that variables appear as arguments of only one or two local functions.222In the same way that any factor graph can be suitably reformulated as a normal factor graph [12], any DE-FG can be suitably reformulated as a DE-NFG. With this, there is no loss in generality in considering only DE-NFGs.

Example 1**.**

Consider the DE-NFG in Fig. 1, which is a pictorial representation of the factorization

[TABLE]

It is called a DE-NFG because some of the edges are double lines that correspond to variables that are paired. (For example, $x_{1}$ and $x^{\prime}_{1}$ are paired in Fig. 1.) Such paired variables are assumed to have both the same alphabet. Moreover, as detailed below, the local functions have to satisfy some constraints.

Definition 2**.**

Consider the factorization

[TABLE]

represented by some DE-NFG. We will use the following conventions:

• •

We call $g$ the global function.

• •

We call $f\in\mathcal{F}$ the local functions. With some abuse of notation, we will also use $f$ to refer to the corresponding function node in the DE-NFG.

• •

For every function node $f\in\mathcal{F}$, the variables associated with the incident double-edges are collected in $\mathbf{x}_{\partial{f}},\mathbf{x}^{\prime}_{\partial{f}}$.

• •

For every function node $f\in\mathcal{F}$, the variables associated with the incident single-edges are collected in $\mathbf{y}_{\delta{f}}$.

Most importantly, we require every local function $f\in\mathcal{F}$ to have the following property:

the local function $f(\mathbf{x}_{\partial{f}},\mathbf{x}^{\prime}_{\partial{f}};\mathbf{y}_{\delta{f}})$ is complex-valued

and is positive semi-definite (PSD).

The latter property is to be understood as follows: for every $\mathbf{y}_{\delta{f}}$ and every complex-valued function $h$ over the alphabet of $\mathbf{x}_{\partial{f}}$ (and with that also over the alphabet of $\mathbf{x}^{\prime}_{\partial{f}}$), it holds that

[TABLE]

(Here and in the following, over-bar denotes complex conjugation.) Clearly, if a function node $f$ has no incident double edges, then the condition in (1) reduces to the condition that the local function $f$ takes on only non-negative real values.

For proving various properties of DE-NFG, the following observation is very beneficial.

Remark 3**.**

For every local function $f\in\mathcal{F}$ and every $\mathbf{y}_{\delta{f}}$, there are a finite set $\mathcal{K}_{f,\mathbf{y}_{\delta{f}}}$ and some complex-valued functions $b_{f,\mathbf{y}_{\delta{f}},k}$ ​, $k\in\mathcal{K}_{f,\mathbf{y}_{\delta{f}}}$ ​, over the alphabet of $\mathbf{x}_{\partial{f}}$ such that

[TABLE]

This follows easily from the eigenvalue decomposition of PSD matrices.

Proposition 4**.**

The partition sum of a DE-NFG, i.e.,

[TABLE]

is a non-negative real number.

Proof.

This can be proven with the help of Remark 3. We omit the details because of space limitations. ∎

As already mentioned, one of the main motivations of the present paper are the NFGs in [9, 10]. So let us show how a “typical” NFG in [9, 10] can be formulated as a DE-NFG.

Example 5**.**

Consider the NFG in Fig. 2 (top), which can be used to do probability computations for the following quantum mechanical setup:

• •

At the beginning, some quantum mechanical system is in some mixed state (represented by the density matrix $\rho$, which is a PSD matrix).

• •

The system then evolves unitarily (represented by $U_{0}$).

• •

Afterwards, a sub-system is measured (represented by measurement operators $\{M_{y}\}_{y}$).

• •

Finally the system evolves unitarily (represented by $U_{1}$).

(For further details, we refer to [9, 10].) This NFG can be turned into the DE-NFG shown in Fig. 2 (bottom) by suitably merging edges (and with that the associated variables) and by suitably defining the DE-NFG’s function nodes. For example, the function node $\tilde{M}$ is defined to be

[TABLE]

Clearly, the function $\tilde{M}(x_{3},x_{4},x^{\prime}_{3},x^{\prime}_{4};y)$ satisfies the required PSD constraint. In fact, the expression in (2) is in the form of the decomposition in Remark 3.

One can check that the redrawing procedure in Example 5 can be applied to all relevant NFGs in [9, 10].

III Sum-Product Algorithm on DE-NFGs

and the Bethe Approximation

In this section we define the SPA for DE-NFGs and discuss some of its properties. In particular, we connect it to generalized versions of the Bethe free energy function.

Once a DE-NFG as in Fig. 1 or in Fig. 2 (bottom) has been defined, we simply consider it as a particular type of NFG and apply the SPA in the standard way [1, 2]. Some comments:

• •

In this paper we only discuss the flooding schedule [1], where all messages are updated at every iteration. Clearly, other update schedules are possible and might be preferable in some cases.

• •

If desired, message can be rescaled by a positive scalar at every iteration.

• •

For reasons of simplicity, we discuss only the case where all edges are full edges, i.e., connect two function nodes. (Note that any DE-NFG can be turned into such a DE-NFG by attaching suitable dummy function nodes to half-edges, thereby turning half-edges into full-edges without changing marginals or the partition sum.)

Recall that in the case of NFGs, messages are functions over the alphabet of the variable associated with an edge. Therefore, along a single-edge $e$ between some function nodes $f$ and $h$, we will have messages $\mu^{(t)}_{e\to f}(y_{e})$ and $\mu^{(t)}_{e\to h}(y_{e})$ at iteration $t$. Similarly, along a double-edge between some function nodes $f$ and $h$, we will have messages $\mu^{(t)}_{e\to f}(x_{e},x^{\prime}_{e})$ and $\mu^{(t)}_{e\to h}(x_{e},x^{\prime}_{e})$ at iteration $t$.

Assumption 6**.**

We make the following assumptions about the initial messages, i.e., about the messages at time $t=0$:

• •

Messages along single-edges are positive real-valued functions.

• •

Messages along double-edges are complex-valued positive definite (PD) functions.

Proposition 7**.**

Let the messages be initialized as in Assumption 6. Then for every iteration $t\geqslant 1$ it holds that:

• •

Messages along single-edges are non-negative real-valued functions.

• •

Messages along double-edges are complex-valued PSD functions.

Proof.

One approach to prove these statements is based on Remark 3. Another approach is based on Schur’s product theorem, which states that the component-wise product of two PSD matrices is a PSD matrix.333Actually, Schur’s product theorem makes the stronger statement that the component-wise product of two PD matrices is a PD matrix. ∎

Definition 8**.**

Consider a collection of SPA messages, one for every edge in both directions. Let

[TABLE]

where $\mathcal{E}$ is the set of all edges, where for every $f\in\mathcal{F}$ we define Z_{f}\triangleq\sum_{\mathbf{x}_{\partial{f}},\mathbf{x}^{\prime}_{\partial{f}},\mathbf{y}_{\delta{f}}}f(\mathbf{x}_{\partial{f}},\mathbf{x}^{\prime}_{\partial{f}};\mathbf{y}_{\delta{f}})\cdot\bigl{(}\prod_{e\in\partial{f}}\mu_{e\to f}(x_{e},x^{\prime}_{e})\bigr{)}\cdot\bigl{(}\prod_{e\in\delta{f}}\mu_{e\to f}(y_{e})\bigr{)}, where for every single-edge $e\in\mathcal{E}$ between function nodes $f$ and $h$ we define $Z_{e}\triangleq\sum_{y_{e}}\mu_{e\to f}(y_{e})\cdot\mu_{e\to h}(y_{e})$, and where for every double-edge $e\in\mathcal{E}$ between function nodes $f$ and $h$ we define $Z_{e}\triangleq\sum_{x_{e},\,x^{\prime}_{e}}\mu_{e\to f}(x_{e},x^{\prime}_{e})\cdot\mu_{e\to h}(x_{e},x^{\prime}_{e})$.

Proposition 9**.**

The function $Z_{\mathrm{Bethe}}$ in Definition 8 has the following properties:

• •

Assume that the messages have the properties in Proposition 7 and assume that $Z_{\mathrm{Bethe}}$ is well-defined, i.e., $Z_{e}\neq 0$ for all $e\in\mathcal{E}$. Then $Z_{\mathrm{Bethe}}$ is a non-negative real number.

• •

Fixed points of the SPA correspond to stationary points of the function $Z_{\mathrm{Bethe}}$. (This generalizes a theorem by Yedidia et al. **[3]**.)

Proof.

Omitted due to space limitations. ∎

Evaluating $Z_{\mathrm{Bethe}}$ in Definition 8 at a fixed point of the SPA results in the Bethe approximation of the partition sum of the DE-NFG.

One can also generalize the Bethe free energy function $F_{\mathrm{Bethe}}$ from [3], where $F_{\mathrm{Bethe}}$ is a function over a suitable generalization of the local marginal polytope. While a statement (analogous to a statement in [3]) that fixed points of the SPA correspond to stationary points of $F_{\mathrm{Bethe}}$ can be made, evaluating $Z_{\mathrm{Bethe}}$ based on $F_{\mathrm{Bethe}}$ is trickier because of the multi-valuedness of the complex logarithm.

IV Examples

In this section we discuss various examples of DE-NFGs. In particular, we compare the obtained Bethe approximation of the partition sum with the true partition sum. (The NFGs in this section have modest sizes so that the true partition function can be computed efficiently.) Moreover, for the first example, we can also make some analytical statements.

Example 10**.**

Let $n$ be some integer larger than one. Consider a DE-NFG whose topology is an $n$-cycle and where all variables take on values in the same finite alphabet $\mathcal{X}$. (Fig. 3 shows such a DE-NFG for $n=4$.) Let $\mathbf{F}$ be a complex-valued PD matrix of size $|\mathcal{X}|^{2}\times|\mathcal{X}|^{2}$ with entries $F(x_{0},x_{1};x^{\prime}_{0},x^{\prime}_{1})$. For $i\in[n]\triangleq\{0,1,\ldots,n\!-\!1\}$, we define the local function $f_{i}$ to be $f_{i}(x_{i},x_{i+1};x^{\prime}_{i},x^{\prime}_{i+1})\triangleq F(x_{i},x_{i+1};x^{\prime}_{i},x^{\prime}_{i+1})$. (All indices are modulo $n$.)

In order to proceed, it is convenient to define the complex-value matrix $\mathbf{B}$ of size $|\mathcal{X}|^{2}\times|\mathcal{X}|^{2}$ with entries $B(x_{0},x^{\prime}_{0};x_{1},x^{\prime}_{1})\triangleq F(x_{0},x_{1};x^{\prime}_{0};x^{\prime}_{1})$ and to define $\tilde{x}_{i}\triangleq(x_{i},x^{\prime}_{i})$, $i\in[n]$.

Let $\mu_{f_{i}\to f_{i+1}}^{(t)}(\tilde{x}_{i+1})$ be the SPA message along the double edge from $f_{i}$ to $f_{i+1}$ at time index $t$. Similarly, let $\mu_{f_{i+1}\to f_{i}}^{(t)}(\tilde{x}_{i+1})$ be the SPA message along the double edge from $f_{i+1}$ to $f_{i}$ at time index $t$. Clearly,

[TABLE]

For $i\in[n]$, we assume the following initializations $\mu_{f_{i}\to f_{i+1}}^{(0)}(\tilde{x}_{i+1})\triangleq\delta(x_{i+1},x^{\prime}_{i+1})$ and $\mu_{f_{i+1}\to f_{i}}^{(0)}(\tilde{x}_{i+1})\triangleq\delta(x_{i+1},x^{\prime}_{i+1})$, where $\delta$ is the Kronecker-delta function.

Because of the properties of the matrix $\mathbf{B}$ that are induced by the properties of the matrix $\mathbf{F}$, the SPA message update rules in (3)–(4) represent so-called completely positive maps (see, e.g., [13]). (For this statement we ignore the rescaling factors.) Using generalizations of Perron–Frobenius theory (see [14, 15]), one can make the following statements:

• •

For every $i\in[n]$, the message $\mu_{f_{i}\to f_{i+1}}^{(t)}(\tilde{x}_{i+1})$ converges to a PD matrix as $t\to\infty$.

• •

For every $i\in[n]$, the message $\mu_{f_{i+1}\to f_{i}}^{(t)}(\tilde{x}_{i+1})$ converges to a PD matrix as $t\to\infty$.

• •

The eigenvalue of the matrix $\mathbf{B}$ with maximum absolute value is a real number and is unique. Let us call it $\lambda_{0}$.

• •

The Bethe approximation of the partition sum is

[TABLE]

Compare this result with the partition sum, which is

[TABLE]

where $\lambda_{0},\ldots,\lambda_{|\mathcal{X}|^{2}-1}$ are the eigenvalues of $\mathbf{B}$. We see that the smaller the ratios \bigl{(}\frac{\lambda_{j}}{\lambda_{0}}\bigr{)}^{n}, $j=1,\ldots,|\mathcal{X}|^{2}\!-\!1$, are, the better the Bethe approximation is.

For $n=4$ and $|\mathcal{X}|=2$, Fig. 3(c) shows the obtained $Z$ and $Z_{\mathrm{Bethe}}$ values for $10^{6}$ experiments based on randomly generating matrices $\mathbf{F}\triangleq\mathbf{U}\cdot\mathbf{D}\cdot\mathbf{U}^{\mathsf{H}}$, which are based on randomly generating unitary matrices $\mathbf{U}$ and diagonal matrices $\mathbf{D}$, where the diagonal entries of $\mathbf{D}$ are sampled i.i.d. from a standard $\chi^{2}$ distribution with one degree of freedom. We see that very often the ratio $Z_{\mathrm{Bethe}}/Z$ is rather close to $1$.

Example 11**.**

Consider now the DE-NFG in Fig. 3(b). For $|\mathcal{X}|=2$, Fig. 3(d) shows the obtained $Z$ and $Z_{\mathrm{Bethe}}$ values for $10^{6}$ experiments based on randomly generating local functions. In contrast to Example 10, where for every instantiation all local function were the same, here for every instantiation all local function are generated independently. We observe the ratio $Z_{\mathrm{Bethe}}/Z$ is reasonably close to $1$, but typically larger than $1$.

Example 12**.**

Let $\mathbf{\theta}$ be a complex-valued matrix of size $n\times n$ with entries $\theta_{i,j}$. The permanent [16] of $\mathbf{\theta}$ is defined to be $\operatorname{perm}(\mathbf{\theta})\triangleq\sum_{\sigma}\prod_{i=1}^{n}\theta_{i,\sigma(i)}$ ​, where the summation is over all $n!$ permutations of the set $[n]\triangleq\{1,\ldots,n\}$. Ryser’s algorithm, one of the most efficient algorithms for exactly computing $\operatorname{perm}(\mathbf{\theta})$ for general matrices $\theta$, requires $\Theta(n\cdot 2^{n})$ arithmetic operations [17], and so the exact computation of permanent is intractable, even for moderate values of $n$. Note that even the computation of the permanent of matrices that contain only zeros and ones is #P-complete [18].

One can formulate an NFG whose partition sum equals $\operatorname{perm}(\mathbf{\theta})$, see, e.g., Fig. 1 in [19]. That NFG is a complete bipartite graph with $n$ function nodes on the left and $n$ function nodes on the right. Here, Fig. 4(a), shows a slightly modified version of that NFG. All variables take values in the set $\mathcal{X}=\{0,1\}$. Moreover, for all $i\in[n]$, the function $f^{\mathrm{L}}_{i}$ is defined to be

[TABLE]

for all $j\in[n]$, the function $f^{\mathrm{R}}_{j}$ is defined analogously; and for all $(i,j)\in[n]^{2}$, the function $f_{i,j}$ is defined to be

[TABLE]

In this example, we consider the following, rather natural generalization to the DE-NFG in Fig. 4(b), where we will use the short-hand $\tilde{x}^{\mathrm{L}}_{i,j}$ for \bigl{(}x^{\mathrm{L}}_{i,j},\tilde{x}^{\mathrm{L^{\prime}}}_{i,j}\bigr{)}, etc. Assume that for $(i,j)\in[n]^{2}$,  $\tilde{\theta}_{i,j}$ is a complex-valued PSD matrix of size $2\times 2$. With this, for $i\in[n]$, the function $\tilde{f}^{\mathrm{L}}_{i}$ is defined to be

[TABLE]

for all $j\in[n]$, the function $\tilde{f}^{\mathrm{R}}_{j}$ is defined analogously; and for all $(i,j)\in[n]^{2}$, the function $\tilde{f}_{i,j}$ is defined to be

[TABLE]

(One can easily verify that these local function define indeed a DE-NFG.) Finally, let $\tilde{Z}$ be the partition sum of this DE-NFG.

This DE-NFG definition has the following two important special cases:

• •

If \tilde{\theta}_{i,j}=\bigl{(}\begin{smallmatrix}1&0\\ 0&\theta_{i,j}\end{smallmatrix}\bigr{)} for all $(i,j)\in[n]^{2}$, then $\tilde{Z}=\operatorname{perm}(\mathbf{\theta})$.

• •

If \tilde{\theta}_{i,j}=\bigl{(}\begin{smallmatrix}1\\ \theta_{i,j}\end{smallmatrix}\bigr{)}\cdot\bigl{(}\begin{smallmatrix}1&\overline{\theta_{i,j}}\end{smallmatrix}\bigr{)}=\Bigl{(}\begin{smallmatrix}1&\overline{\theta_{i,j}}\\ \theta_{i,j}&|\theta_{i,j}|^{2}\end{smallmatrix}\Bigr{)} for all $(i,j)\in[n]^{2}$, then \tilde{Z}=\operatorname{perm}(\mathbf{\theta})\cdot\operatorname{perm}(\overline{\mathbf{\theta}})=\bigl{|}\operatorname{perm}(\mathbf{\theta})\bigr{|}^{2}, where $\overline{\mathbf{\theta}}$ denotes the matrix whose entries are the complex-conjugate values of the entries of $\mathbf{\theta}$. (Note that such partition sums are of interest in quantum information processing [20], where $\mathbf{\theta}$ are certain types of square matrices over the complex numbers. We refer to [20] for details.)

In our experiments, we considered the following setup. Namely, for every $(i,j)\in[n]^{2}$, we independently generate $\tilde{\theta}_{i,j}$ as follows: $\tilde{\theta}_{i,j}(0,0)\triangleq 1$;  $\tilde{\theta}_{i,j}(1,0)$ is picked uniformly from the unit circle in the complex plane;  $\tilde{\theta}_{i,j}(0,1)\triangleq\overline{\tilde{\theta}_{i,j}(1,0)}$;  $\tilde{\theta}_{i,j}(1,1)$ is picked uniformly (and independently of the other entries) from the real line interval $[1.10,\ 11.10]$. Fig. 4(c) shows the obtained $Z$ and $Z_{\mathrm{Bethe}}$ values for $5000$ experiments for the case $n=5$. We observe that the ratio $Z_{\mathrm{Bethe}}/Z$ is concentrated around a value smaller than $1$.

V Connections to a Paper by Mori

Finally, let us point out that there are strong connections of DE-NFGs to the setup in Section V of a recent paper by Mori [21]. Assume to have a bipartite DE-NFG. (Such a DE-NFG can always be obtained by suitably inserting dummy function nodes.) Then the partition sum can be written as some inner product between, on the one hand, the tensor product of the local functions corresponding to first class of function nodes of the bipartite DE-NFG, and, on the other hand, the tensor product of the local functions corresponding to the second class of function nodes of the bipartite DE-NFG. Once this connection is observed, one can translate Mori’s results (like loop calculus expansions) to DE-NFGs.

VI Conclusion

In this paper we have defined DE-NFGs and studied some of their properties. In particular, we have shown some promising numerical studies of the Bethe approximation to the partition sum. Many open questions remain. For example, can some of the results in [19] be generalized to the setup in Example 12? Or, as in the context of computing the pattern maximum likelihood estimate, which can be formulated as optimizing the parameters of some graphical model toward maximizing the partition function, and where the Bethe partition sum was beneficially used as a surrogate function [22], can the Bethe partition sum of a DE-NFG serve as a suitable surrogate function in some partition function optimization problem?