Integer programming on the junction tree polytope for influence diagrams
Axel Parmentier1, Victor Cohen1, Vincent Leclère1, Guillaume Obozinski2, Joseph Salmon3
(1Université Paris-Est, CERMICS (ENPC), F-77455 Marne-la-Vallée, France
2 Swiss Data Science Center, EPFL & ETH Zürich, Switzerland
3 IMAG, Univ Montpellier, CNRS, Montpellier, France
)
Abstract
Keywords: Influence diagrams, Partially Observed Markov Decision Processes, Probabilistic graphical models, Linear Programming.
Influence Diagrams (ID) are a flexible tool to represent discrete stochastic optimization problems, including Markov Decision Process (MDP) and Partially Observable MDP as standard examples.
More precisely, given random variables considered as vertices of an acyclic digraph, a probabilistic graphical model defines a joint distribution via the conditional distributions of vertices given their parents.
In ID, the random variables are represented by a probabilistic graphical model whose vertices are partitioned into three types : chance, decision and utility vertices.
The user chooses the distribution of the decision vertices conditionally to their parents in order to maximize the expected utility.
Leveraging the notion of rooted junction tree, we present a mixed integer linear formulation for solving an ID, as well as valid inequalities, which lead to a computationally efficient algorithm.
We also show that the linear relaxation yields an optimal integer solution for instances that can be solved by the “single policy update”, the default algorithm for addressing IDs.
1 Introduction
In this paper we want to address stochastic optimization problems with structured information and discrete decision variables, via mixed integer linear reformulations.
We start by recalling the framework of influence diagrams (more details can be found in (Koller and Friedman, 2009, Chapter 23)), and present the classical linear formulation for some special cases.
1.1 The framework of parametrized influence diagram
Let G=(V,E) be a directed graph, and, for each vertex v in V, let Xv be a random variable taking value in a finite state space Xv. For any C⊂V, let XC denote (Xv)v∈C and XC be the cartesian product XC=∏v∈CXv.
We say that the distribution of the random vector XV factorizes as a directed graphical model on G if, for all xV∈XV, we have
[TABLE]
where prt(v) is the set of parents of v, that is, the set of vertices u such that (u,v) belongs to E, and pv∣prt(v)(xv∣xprt(v))=P(Xv=xv∣Xprt(v)=xprt(v)).
Further, given an arbitrary collection of conditional distributions
{pv∣prt(v)}v∈V,
Equation (1) uniquely defines a probability distribution on XV.
Let (Va,Vc,Vr)
be a partition of V where Vc is the set of chances vertices, Va is the set of decision vertices, and Vr is the set of utility vertices (the ones with no descendants). For ease of notation we denote Vs=Vc∪Vr.
Letters a, r, and s respectively stand for action, reward, and state in Va, Vr, and Vs. We say that G=(Vs,Va,E) is an Influence Diagram (ID).
Consider a set of conditional distributions p={pv∣prt(v)}v∈Vc∪Vr, and a collection of reward functions r={rv}v∈Vr, with rv:Xv→R.
Then we call (G,XV,p,r) a Parametrized Influence Diagram (PID).
We will sometimes refer to the parameters (XV,p,r) by ρ for conciseness.
Let Δv denote the set of conditional distributions δv∣prt(v) on Xv given Xprt(v).
Given the set of conditional distributions p,
a policy δ in Δ=∏v∈VaΔv,
uniquely defines a distribution Pδ on XV through
[TABLE]
Let Eδ denote the corresponding expectation.
The Maximum Expected Utility (MEU) problem associated to the
PID (G,XV,p,r) is the maximization problem
[TABLE]
A deterministic policy δ∈Δd⊂Δ, is such that for every v∈Va, and any xv,xprt(v)∈Xv×Xprt(v), δv∣prt(v)(xv∣xprt(v)) is a Dirac measure.
It is well known that there always exists an optimal solution to MEU (3) that is deterministic (see e.g., (Liu, 2014, Lemma C.1) for a proof).
We conclude this section with some classical examples of IDs, shown in Figure 1.
Example 1*.*
Consider a maintenance problem where at time t a machine is in state st.
The action at taken by the decision maker according to the current state is typically maintaining it (which is costly) or not (which increases the probability of failure).
State and decision together lead to a new (random) state st+1, and the triple (st,at,st+1) induces a reward rt.
This is an example of a Markov decision process (MDP) which is probably the simplest ID,
represented in Figure 1(a).
In practice, the actual state st of the machine is often not known, but we only have some observation ot carrying partial information about the state,
which leads to a more complex ID known as a partially observed Markov decision process (POMDP).
In theory, an optimal decision should be taken knowing all past observations and decisions (which is the perfect recall case).
However, this would lead to policies living in spaces of exponentially large dimension and untractable MEU problems.
It is thus common to restrict the decision at to be made only based on observation ot,
as illustrated in Figure 1(b).
∎
Example 2*.*
Consider two chess players : Bob and Alice.
They are used to play chess and for each game they bet a symbolic coin.
However, they can refuse to play.
Suppose that Alice wants to play chess every day.
On the day t, she has a current confidence level st.
The day of the game, her current mental fitness is denoted ot.
When Bob meets with Alice, he makes the decision to play depending on her demeanor, denoted ut. Then Bob can accept or decline the challenge, and his decision is denoted at.
Let vt denote the winner (getting a reward rt). If Bob declines the challenge, there is no winner and no reward. Then, Alice’s next confidence level is affected by the result of the game and her previous confidence level.
This stochastic decision problem can be modeled by an influence diagram as shown in Figure 2.
∎
1.2 Solving MDP through linear programs
We recall here a well known linear programming formulation for MDP (see e.g., Puterman (2014)), which is a special case of the Mixed Integer Linear Program (MILP) formulation introduced in the paper.
We denote by p(s′∣s,a) the probability of transiting from state s to state s′ if action a is taken, and r(s,a,s′) the reward associated to this transition.
For t∈[T]:={1,…,T}, let μst represent the probability of being in state s at time t, let μsat represent the probability of being in state s and taking action a at time t, and let μsas′t represent the probability of being in state s and taking action a at time t, while transiting to state s′ at time t+1.
This leads to the following mixed integer linear program
[TABLE]
where the objective (4a) is simply the expected reward, Constraints (4b) represent the state dynamics, Constraints (4c) set the initial state of the system to s0, and Constraints (4d)-(4f) ensure that μ represent marginals laws of a joint distribution. Integrity constraints (4f) ensure that the policy chosen is deterministic.
In the MDP case, we can drop these integrity constraints and still obtain an optimal solution. In Section 4, the integrity constraints will come out to be useful in the general case.
1.3 Literature
Influence diagrams were introduced by Howard and Matheson (1984) (see also Howard and Matheson, 2005)
to model stochastic optimization problems using a probabilistic graphical model framework.
Originally, the decision makers were assumed to have perfect recall (Shenoy, 1992; Shachter, 1986; Jensen et al., 1994) of the past actions family.
Lauritzen and Nilsson (2001) relaxed this assumption111These authors used the name limited memory influence diagrams when relaxing the perfect recall assumption, but we follow the convention of Koller and Friedman (2009) who still call them influence diagrams (ID).
and provided a simple (coordinate descent) algorithm to find a good policy: the Single Policy Update (SPU) algorithm.
The same authors also introduced the notion of soluble ID as a sufficient condition for SPU to converge to an optimal solution.
This notion has been generalized by Koller and Milch (2003) to obtain a necessary and sufficient condition.
In general, SPU only finds a locally optimal policy, and requires to perform exact inference, so that it is therefore limited by the treewidth (Chandrasekaran et al., 2008).
More recently, Mauá and Campos (2011) and Mauá and Cozman (2016) have introduced a new algorithm, Multiple Policy Update, which has both an exact and a heuristic version and relies on dominance to discard partial solutions. It can be interpreted as a generalization of SPU where several decisions are considered simultaneously.
Later on, Khaled et al. (2013) proposed a similar approach, with a Branch-and-Bound flavor, while Liu (2014) introduced heuristics based on approximate variational inference.
Finally, Maua (2016) has recently shown that the problem of solving an ID can be polynomially transformed into a maximum a posteriori (MAP) problem, and hence can be solved using popular MAP solvers such as toulbar2 (Hurley et al., 2016).
Finding an optimal policy for an ID has been shown to be NP-hard even when restricted to IDs of treewidth non-greater than two, or to trees with binary variables Mauá et al. (2012a, 2013).
Note that even obtaining an approximate solution is also NP-hard Mauá et al. (2012a).
Credal networks are generalizations of probabilistic graphical models where the parameters of the model are not known exactly. MILP formulations for credal networks that could be applied to IDs have been introduced by de Campos and Cozman (2007); de Campos and Ji (2012).
However, the number of variables they require is exponential in the pathwidth, which is non-smaller and can be arbitrarily larger than the width of the tree we are using (follows from (Scheffler, 1990, Theorem 4)),
and the linear relaxation of their MILP is not as good as the one of the MILP we propose, and does not yield an integer solution on soluble IDs.
Our approach can naturally be extended to credal networks.
1.4 Contributions
The contributions of the paper are as follows.
We introduce a non-linear program and a mixed integer linear program for the MEU problem on influence diagrams.
These mathematical programs rely on a variant of the concept of a strong junction tree which we introduce and call a rooted junction tree.
We provide algorithms to build rooted junction trees that lead to “good” mathematical programs for influence diagrams.
We introduce a particular form of valid cuts for the obtained mixed integer linear program.
These valid cuts leverage conditional independence properties in the influence diagram.
We show that our cuts are the strongest ones in a certain sense.
We believe that this idea of leveraging conditional independence to obtain valid cuts is fairly general and could be extended to other contexts.
We establish a link between the linear relaxation of our MILP and the concept of soluble relaxation previously introduced in the literature on influence diagrams.
In fact, our relaxation provides a better bound than those relaxations.
We provide two new characterizations of soluble influence diagrams.
First, as the only influence diagrams that can be solved to optimality using the linear relaxation of our mixed integer linear program.
Second, and more importantly, as the influence diagrams for which there exists a rooted junction tree such that the set of collections of moments of distributions that are induced by the different policies is convex.
We illustrate our mathematical programs and their properties on some simple numerical examples.
1.5 Organization of the paper
In Section 2, we recall some definitions for graphical models, that are used
to extend the notion of junction tree to rooted junction tree in Section 3.
With these tools, Section 4 introduces a bilinear formulation that can be rewritten as a mixed integer linear programming (MILP) formulation to the MEU Problem (3).
In Section 5 we give efficient valid cutsfor the MILP formulation, and interpret them in terms of graph relaxations.
Section 6 studies the polynomial case of soluble ID, showing that the ID that can be solved to optimality
by SPU can be solved by (continuous) linear programming using our formulation.
Finally Section 7 summarizes our numerical experiments.
2 Tools from Probabilistic graphical model theory
In this section we present notations and tools used in the following sections to refomulate the MEU Problem 3.
2.1 Graph notation
This section introduces our notations for graphs, which are for the most part the ones commonly used in the combinatorial optimization community (Schrijver, 2003).
A directed graph G is a pair (V,E) where V is the set of vertices and E⊆V2 the set of arcs.
We write u→v when (u,v)∈E. Let [k]:={1,…,k}.
A path is a sequence of vertices v1,…,vk such that vi→vi+1, for any i∈[k−1]. A path between two vertices u and v is called a u-v path.
We write
u\leavevmode\hbox to20.32pt{\vbox to17.03pt{\pgfpicture\makeatletter\hbox{\hskip 3.33301pt\lower-3.33301pt\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox to0.0pt{\pgfsys@beginscope\pgfsys@invoke{ }{{}}{{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{
{}{}}}{
{}{}}
{{}{{}}}{{}{}}{}{{}{}}
{
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{0.0pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{
{}{}}}{
{}{}}
{{}{{}}}{{}{}}{}{{}{}}
{
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{13.6575pt}{0.0pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
{}{{}}{}
{}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@setdash{3.0pt,3.0pt}{0.0pt}\pgfsys@invoke{ }{}{}{}{}{{}}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@lineto{10.05756pt}{0.0pt}\pgfsys@stroke\pgfsys@invoke{ }{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{10.05756pt}{0.0pt}\pgfsys@invoke{ }\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{
{}{}}}{
{}{}}
{{}{{}}}{{}{}}{}{{}{}}
{
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{ }\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{2.89749pt}{3.533pt}\pgfsys@invoke{ }\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{ }\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{{G}}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}v to denote the existence of a u-v path in G, or simply u\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpicturev when G is clear from context.
We write u⇌v if there is an arc u→v or v→u.
A trail is a sequence of vertices v1,…,vk such that vi⇌vi+1, for all i∈[k−1].
A parent (resp. child) of a vertex v is a vertex u such that (u,v) (resp. (v,u)) belongs to E; we denote by prt(v) the set of parents vertices (resp. cld(v) the set of children vertices).
The family of v, denoted by fa(v), is the set {v}∪prt(v).
A vertex u is an ascendant (resp. a descendant) of v if there exists a u-v path. We denote respectively asc(v) and dsc(v) the set of ascendants and descendants of v.
Finally, let asc(v)={v}∪asc(v), and dsc(v)={v}∪dsc(v).
For a set of vertices C, the parent set of C, again denoted by prt(C), is the set of vertices u that are parents of a vertex v∈C.
We define similarly fa(C), cld(C), asc(C), and dsc(C).
Note that we sometimes indicate in subscript the graph according to which the parents, children, etc., are taken.
For instance, prtG(v) denotes the parents of v in G.
We drop the subscript when the graph is clear from the context.
A cycle is a path v1,…,vk such that v1=vk.
A graph is connected if there exists a path between any pair of vertices.
An undirected graph is a tree if it is connected and has no cycles.
A directed graph is a directed tree if its underlying undirected graph is a tree.
A rooted tree is a directed tree such that all vertices have a common ascendant referred to as the root of the tree222The probabilistic graphical model community sometimes calls a directed tree what we call here a rooted tree, and a polytree what we call here a directed tree..
In a rooted tree, all vertices but the root have exactly one parent.
2.2 Directed graphical model
In this paper, we manipulate several distributions on the same random variables. Given three random variables X, Y, Z, the notation
\big{(}X\,\bot\!\!\!\bot\,Y|Z\big{)}_{\mu}
stands for “X is independent from Y given Z according to μ”. The parenthesis (⋅)μ are dropped when μ is clear from context.
The same notation is used for independence of events.
A well-known sufficient condition for a distribution to factorize as a probabilistic graphical model is that each vertex is independent from its non-descendants given its parents.
Proposition 1**.**
(Koller and Friedman, 2009, Theorem 3.1, p. 62)*
Let Pμ be a distribution on XV.
Then Pμ factorizes as a directed graphical model on G, that is*
[TABLE]
if and only if
[TABLE]
2.3 Junction trees
When dealing with the MEU Problem 3, one needs to deal with distributions μV on XV that factorize as in (2) for some policy δ.
In theory, it suffices to consider distributions μV satisfying the conditional independences given by Equation 5 and such that Pμ(Xv∣Xprt(v))=pv∣prt(v) for each vertex v that is not a decision.
However, the joint distribution μV on all the variables is too large to be manipulated in practice as soon as V is moderately large. In that case, it is handy to work with a vector of moments τ=(τC)C∈V, where V⊆2V, that is, a vector of distributions τC on subsets of variables C of tractable size.
A vector of moment (τC)C∈V derives from a distribution μV on XV if each moment τC∈[0,1]XC is the marginal of μV, i.e., τC(xC)=∑xV\C∈XV\CμV(xC,xV\C) for all C in V and xC in XC. To keep notations light, we will write this type of equality more compactly as τC=∑xV\CμV.
We use the notation μ=(μC)C∈V for the vector of moments deriving from a distribution, and Pμ or μV for the corresponding distribution on XV.
A necessary condition for a vector of moments (τC)C∈V to derive from a distribution is to be locally consistent, that is to induce the same marginals on the intersections of pairs of elements of V, i.e., that for all C1,C2∈V, we have
[TABLE]
where, as before, ∑xC1\C2τC1 is the vector \big{(}\sum_{x_{C_{1}\backslash C_{2}}\in\mathcal{X}_{C_{1}\backslash C_{2}}}\tau_{C_{1}}(x_{C_{1}\backslash C_{2}},x_{C_{1}\cap C_{2}})\big{)}_{x_{C_{1}\cap C_{2}}\in\mathcal{X}_{C_{1}\cap C_{2}}}.
It turns out that graphical model theory provides a condition on the choice of V together with the choice of local consistency constraints which are sufficient for (τC)C∈V to derive from a distribution on XV. This is done via the definition of a junction tree.
Let G=(V,A) be an undirected graph associated with G=(V,E) with V⊆2V, and such that there is a mapping v↦Cv from V to V satisfying that fa(v)⊆Cv.
If G is a tree, and satisfies the running intersection property, i.e., that given two vertices C1 and C2 in V, any vertex C on the unique undirected path from C1 to C2 in G satisfies C1∩C2⊂C, then G is called a junction tree of G.
See Figure 3 for an illustration of this notion.
Given a junction tree G, its associated marginal polytope LG0 is defined as follows
[TABLE]
Then τ=(τC)C∈V is a vector of moments deriving from a distribution μV on XV if and only if τ∈LG0 (Wainwright and Jordan, 2008, Proposition 2.1).
3 Rooted junction trees
To solve the MEU Problem (3), we work on vectors of moments (μC)C∈V that correspond to the moments of distributions μ induced by policies δ∈Δ.
Hence, we are interested in vectors μ of moments such that μV factorizes as a directed graphical model on G.
Such vectors of moments necessarily satisfy a “local” version of the sufficient condition (5), which is that for τC=μC,
[TABLE]
Given a vector of moment τC in the local polytope of a junction tree (V,A), satisfying (7) is not a sufficient condition for τC to be the moments of a distribution μV that factorizes on G.
But it becomes a sufficient condition under the additional assumption that (V,A) is a “rooted junction tree”, a notion that we introduce in this section, and develop in more detail in Appendix A.
3.1 Definition and main properties
Let G=(V,E) be a junction tree on G=(V,A) and v∈V a vertex of G.
Then, thanks to the running intersection property, the subgraph Gv of G made of all nodes C∈V containing v is a tree.
Moreover, any orientation of the edges of G that makes it a rooted tree, also makes Gv a rooted tree, and we denote Cv its root node.
Definition 1**.**
A rooted junction tree (RJT)
on G=(V,E) is
a rooted tree with nodes in 2V, such that
its underlying undirected graph G=(V,A) is a junction tree,
for all v∈V, we have fa(v)⊆Cv,
where Cv is the root clique of v defined as the root node of
the subgraph Gv of G induced by the nodes C∈V containing v.
Let G be an RJT on G, and v a vertex of V. Given C∈G, let C˚={v∈V:Cv=C}, which we call the offspring of C, and let Cˇ denote C\C˚.
See Figure 3 for a graphical example of this notion.
Note that an RJT always exists: Indeed, the cluster graph composed of a single vertex C=V is an RJT.
Algorithms to build interesting RJT are provided in Section 3.2.
Theorem 2, which is a natural generalization of the well-known Proposition 1, ensures that given a vector of moments on an RJT that satisfies local independences, we can construct a distribution on the initial directed graphical model which admits these moments as marginals.
Theorem 2**.**
Let μ be a vector of moments in the local polytope of an RJT G on G=(V,E).
Suppose that for each vertex v, according to μCv, the variable Xv is independent from its non-descendants in G that are in Cv, conditionally to its parents.
Then there exists a distribution Pμ on XV factorizing on G with moments μ.
Remark 1*.*
By adding nodes to an RJT, we can always turn it into an RJT satisfying Cv˚={v} for each vertex v in Va. Indeed, suppose that C˚={v1,…,vk}, where v1,…,vk are given along a topological order.
It suffices to replace the node C by C1→C2→⋯→Ck, where Ci=C\{vi+1,…,vk}. Note that for such RJTs we have Cˇv=Cv\{v}.
∎
Remark 2*.*
Jensen et al. (1994, beginning of Section 4) introduces a similar notion of strong junction tree.
It relies on the notion of elimination ordering for a given influence diagram with perfect recall.
The main difference is that a strong junction tree is a notion on an influence diagram, where the set of decision vertices and their orders play a role, when RJTs rely on the underlying digraph.
The notion of strong junction tree is obtained by replacing (ii) in the definition of an RJT by: “given an elimination ordering, if (Cu,Cv) is an arc, there exists an ordering of Cv that respects the elimination ordering such that Cu∩Cv is before Cv\Cu in that ordering”.
An RJT is a strong junction tree.
Conversely, a strong junction tree is not necessarly an RJT. Indeed, Jensen et al. (1994, Figure 4) shows an example of strong junction where there is v∈V such that fa(v)⊊Cv.
As strong junction trees is a notion on influence diagram and not on graphs, Theorem 2 has no natural generalization for strong junction trees.
∎
3.2 Building an RJT
Although ({V},∅) is a rooted junction tree, the concept has only practical interest if it is possible to construct RJTs with small cluster nodes.
In that respect, note that any RJT must satisfy, for all u,v∈V, the implication
[TABLE]
where C\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureC′ denotes the existence of a C-C′ path in the RJT G considered. This notation will be used throughout this section.
Indeed, since u∈Cu and fa(w)⊂Cw by definition, and since Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw, the running intersection property implies u∈Cv.
This motivates Algorithm 1, a simple RJT construction algorithm which propagates iteratively elements present in each cluster node to their parent cluster node.
Let ⪯ be an arbitrary topological order on G, and max⪯C denote the maximum of C for the topological order ⪯.
The algorithm maintains a set Cv′
for each vertex v, which coincide at the end of the algorithm with the nodes Cv in the RJT produced.
We denote by Cˇv′ is the set Cv′\{v}.
As we will show, Algorithm 1 produces an RJT G=(V,A) which is minimal for ⪯, in the sense that it satisfies a converse of (8).
Remark 3*.*
Algorithm 1 takes as input a topological order on G.
For a practical use, we recommend to use Algorithm 3 in Appendix C, which builds simultaneously the RJT and a “good” topological order.
∎
For instance, for any topological order on the graph of the chess example of Figure 2, Algorithm 1 produces the RJT illustrated on Figure 4.
The following proposition, whose proof can be found in Appendix A shows that Algorithm 1 builds the minimal RJT.
Proposition 3**.**
Algorithm 1 produces an RJT such that the root node Cv of v is Cv′, satisfying Cv˚={v}, that admits ⪯ as a topological order, and such that (u∈Cv)⇒(u⪯v). Moreover, its cluster nodes are minimal in the sense that
[TABLE]
4 MILP formulation for influence diagrams
Given that Algorithm 1 produces an RJT such that Cv˚={v} for all v∈V, we will assume in the rest of the paper that all the RJTs considered satisfy this property.
As noted in Remark 1, any RJT can be turned into an RJT satisfying this property by adding more nodes.
In the rest of the paper, we work with the following variant of the local polytope LG0 defined in Equation (6)
[TABLE]
where moments μCˇv have been introduced. This is for convenience, and all the results could have been written using LG0.
On graphical models, the inference problem, which is hard in general, becomes easy on junction trees.
Since problem (3) is NP-hard even when restricted to graphs of treewidth 2 (Mauá et al., 2012b),
unless P=NP, the situation is strictly worse for the MEU problem associated with influence diagrams.
However, we will see in this section that, given a rooted junction tree, we can obtain mathematical programs to solve the MEU problem 3 with a tractable number of variables and constraints provided that cliques are of reasonable size.
We first obtain an NLP formulation in Section 4.1, and then linearize it into an exact mixed integer linear program (MILP) in Section 4.2.
4.1 An exact Non Linear Program formulation
Consider a Parameterized Influence Diagram (PID) encoded as the quadruple (G,X,p,r), where G=(V,E) is
a graph with set of vertices V partitioned into (Va,Vs), with
X=∏v∈VXv the support of the vector of random variables attached to all vertices of G,
p={pv∣prt(v)}v∈Vs is the collection of fixed and assumed known conditional probabilities,
and r={rv}v∈Vr is the collection of reward functions333we remind the reader that Vr is the set of utility vertices as introduced in Section 1.1
rv:Xv→R which we will also view as vectors rv∈R∣Xv∣.
For (G,X,p,r) a given PID, and G a given RJT,
we introduce the following polytope
[TABLE]
where the equality μCv=μCˇvpv∣prt(v) should be understood functionally, i.e., meaning that μCv(xCv)=μCˇv(xCˇv)pv∣prt(v)(xv∣xprt(v)),∀xCv∈XCv; we will use such functional (in)equalities throughout the paper.
We omit the dependence of P in (G,X,p,G) when the context is clear.
Consider the following Non Linear Program (NLP)
[TABLE]
where the inner product notation ⟨rv,μv⟩ stands for ∑xvμv(xv)rv(xv).
Note that the constraints δ∈Δ are implied by the other ones.
Theorem 4**.**
The (NLP) Problems (11) and (14) are equivalent to the MEU Problem (3),
in the sense that they have the same value and that, if (μ,δ) is a feasible solution for Problem (11), then δ defines an admissible policy for Problem (3), and μ characterizes the moments of the distribution induced by δ.
Proof.
If (μ,δ) is a solution of (11), then μ is is a solution of (14), and conversely, if μ is a solution of (14), by definition of S(G), there exists δ such that (μ,δ) is a solution of (11), which gives the equivalence between (11) and (14).
Let now (μ,δ) be an admissible solution of Problem (11). Then δ is an admissible solution of the MEU problem. We now prove that μ corresponds to the moments of the distribution Pδ induced by δ, from which we can deduce that \mathbb{E}_{\delta}\Big{(}\sum_{v\in V^{\mathrm{r}}}r_{v}(X_{v})\Big{)}=\sum_{v\in V^{\mathrm{r}}}\langle r_{v},\mu_{v}\rangle.
Note that, if A, P, and D are disjoint subsets of V, μ is a distribution on XV, μA∪P∪D is the distribution induced by μ on XA∪P∪D, and pD∣P is a conditional distribution of D given P,
then
[TABLE]
where the independence is according to μ.
By (12), we have that the vector μ satisfies the conditions of Theorem 2, and hence corresponds to a distribution Pμ that factorizes on G. Furthermore,
constraint (10)
ensures that Pμ(Xv∣Xprt(v))=pv∣prt(v) for all v∈Vs, which yields the result.
Conversely, let δ be an admissible solution of the MEU Problem (3),
and μ be the vector of moments induced by Pδ. We have μCv=μCˇvpv∣prt(v) for v in Vs
and μCv=μCˇvδv∣prt(v) for v in Va,
and (μ,δ) is a solution of (11).
Furthermore, \mathbb{E}_{\delta}\Big{(}\sum_{v\in V^{\mathrm{r}}}r_{v}(X_{v})\Big{)}=\sum_{v\in V^{\mathrm{r}}}\langle r_{v},\mu_{v}\rangle, and (11) is equivalent to the MEU Problem (3).
∎
By introducing the following set of moments
[TABLE]
we can reformulate the Problem (11) more concisely as
[TABLE]
S(G) is the set of moments corresponding to distributions induced by feasible policies: μ is in S(G) if there exists δ in Δ such that μCv(xCv)=Pδ(XCv=xCv) for all v
and xCv.
It is non-convex in general as shown by the examples in the proof of Theorem 12.
However, we show in Section 6 that S(G) is a polytope if G is soluble,
a property identifying “easy” IDs.
4.2 MILP formulation
The NLP (11) is hard to solve due to the non-linear constraints (11d).
But by Theorem 4, Problems (3) and (11) are equivalent, and in particular admit the same optimal solutions in terms of δ.
We recall that there always exists at least one optimal policy which is deterministic (and therefore integral) for Problem (3), that is a policy δ such that
[TABLE]
We can therefore add integrality constraint (15) to (11).
With this integrality constraint, Equation (11d) becomes a logical constraint, *i.e., *a constraint of the form λy=z with λ binary and continuous y and z.
Such constraints can be handled by modern MILP solvers such as CPLEX or Gurobi, that can therefore directly solve Problem (11).
Alternatively, by a classical result in integer programming, we can turn Problem (11) into an equivalent MILP by replacing constraint (11d) by its McCormick relaxation (McCormick, 1976).
For a given p, let b be a vector of upper bounds bCˇv(xCˇv) satisfying
[TABLE]
For such a vector b,
we say that, for a given node v, (μCv,δv∣prt(v)) satisfies McCormick’s inequalities (see appendix D) if
[TABLE]
Note that the last inequality μCv≤μCˇv can be omitted in our case as it is implied by the marginalization constraint μCˇv=∑xvμCv in the definition of LG.
Given the upper bounds provided by b, we introduce the polytope of valid moments and decisions satisfying all McCormick constraints:
[TABLE]
With the previously introduced notation the MEU Problem (3) is equivalent to the following MILP:
[TABLE]
where Δd is the set of deterministic policies and contains the integrality constraints (15).
Remark 4*.*
The strength of the McCormick constraints (McCormick(v,b)) depends on the quality of the bounds bCˇv on μCˇv.
As for a solution μ of Problem (18), μCˇv corresponds to a probability distribution,
the simplest admissible bound over μCˇv is simply b=1.
Unfortunately, McCormick’s constraints are loose in this case:
we show in Appendix D.2.1 that, for any μ in P, there exists δ in Δ such that (μ,δ) satisfies the McCormick constraints.
Hence, when b=1, McCormick constraints fail to retain any information about the conditional independence statements encoded in the associated nonlinear constraints.
Since δ does not appear outside of the McCormick constraints, their sole interest in that case is to enable the branching decisions on δ to have an impact on μ.
Appendix D.2.2 gives an example showing that McCormick constraints do retain information about the conditional independence if bounds bCˇv smaller than 1 are used.
Finally,
Appendix D.3 provides a dynamic programming algorithm that efficiently computes such a b.∎
5 Valid cuts
Classical techniques in integer programming such as branch and bound algorithms rely on solving the relaxation of the MILP to obtain a lower bound on the value of the objective.
For Problem (18) the relaxation is likely to be poor, and so the MILP is not well solved by off-the-shelf solvers: indeed as explained above, when b=1, the McCormick inequalities fail completely to enforce in the linear relaxation the conditional independences that are encoded in the nonlinear constraints, and using a better bound b does not completely adress the issue.
In this section, we introduce valid cuts to strengthen the relaxation and ease the MILP resolution.
A valid cut for a MILP is an (in)equality that is satisfied by any solution of the MILP, but not necessarily by solutions of its linear relaxation.
A family of valid cuts is stronger than another when the former yields a polytope strictly included in the latter.
5.1 Constructing valid cuts
By restricting ourselves to vectors of moments μ∈P, we have imposed
[TABLE]
because μ∈P must satisfy μCv=μCˇvpv∣prt(v).
If we could impose as well the nonlinear constraints μCv=μCˇvδv∣prt(v) for v in Va, we would be able to impose that decisions encoded in μ at the nodes a∈Va satisfy Pμ(Xa∣XCa\{a})=Pμ(Xa∣Xprt(a)). Unfortunately, in general, The constraint μCv=μCˇvpv∣prt(v) for v in Vs is linear only because pv∣prt(v) is a constant that does not depend on δ.
But, as an indirect consequence of setting the conditional distributions pv∣prt(v) for v∈Vs, there are other conditional distributions that do not depend on δ.
Indeed, for some pairs of sets of vertices C,D with D⊆C, the conditional probabilities Pδ(XD=xD∣XC\D=xC\D) are identical for any policy δ. We can therefore introduce valid cuts of the form
[TABLE]
While these additional constraints are not needed to set the value of the conditionals on v∈Vs and the conditional independences of the form Xv⊥⊥XV\dsc(v)∣Xprt(v) for v∈Vs, they can be useful to enforce some of the conditional independences that should be satisfied by μ at decision nodes. In particular, if there exists a subset M of C\D such that pD∣C\D=pD∣M, then (19) enforces that for any v∈Va∩(C\(D∪M)), we have Pμ(Xa∣XD∪M)=Pμ(Xa∣XM).
Clearly, the larger D, the stronger the valid cut. This motivates the following definition.
Definition 2**.**
Given a set of vertices C, we define C⊥⊥ to be the largest subset D of C such that, for any parametrization of G,
there exists pD∣C\D such that Pδ(XD∣XC\D)=pD∣C\D holds for any policy δ.
We define C^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}} as C\C⊥⊥.
It is not obvious that a largest such set exists and is unique, and therefore that C⊥⊥ is well defined. We prove that it is the case later in this section.
As for now, if we accept that C⊥⊥ is well defined,
then the equalities
[TABLE]
are the strongest valid cuts of the form (19) that we can obtain for Problem (18).
We can then define P⊥⊥ as the polytope we obtain when we strengthen P with our valid cuts:
[TABLE]
In the definition of P⊥⊥, we decided to introduce valid cuts of the form (20) only for sets of vertices C of the form Cv with v∈Va. This is to strike a balance between the number of constraints added and the number of independences enforced.
Our choice is however heuristic, and it could notably be relevant to introduce constraints of the form (20) for well chosen C⊊Cv.
Figure 5 provides an example of ID where valid cuts (20) reduce the size of the initial polytope. To compute C⊥⊥, we have used the characterization in the next section.
5.2 Characterization of C⊥⊥
In order to characterize C⊥⊥, we need some concepts from graphical model theory. The first notions make it possible to identify conditional independence from properties of the graph.
Let D⊂V be a set of vertices. A trail v1⇌⋯⇌vn is active given D if,
whenever there is a v-structure vi−1→vi←vi+1,
then vi or one of its descendant is in D, and no other vertex of the trail is in D.
Two sets of vertices B1 and B2 are said to be d-separated by D in G, and we will denote this property by B1⊥B2∣D, if there is no active trail between B1 and B2 given D.
We have XB1⊥⊥XB2∣XD
for any distribution that factorizes on G
if and only if B1 and B2 are d-separated by D (Koller and Friedman, 2009, Theorem 3.4).
The other notion we need is the one of augmented model (Koller and Friedman, 2009, Chapter 21).
Consider (G,ρ), a PID with G=(Vs,Va,E), and let V=Va∪Vs. For each v∈Va, we introduce a vertex ϑv and a corresponding random variable θv.
The variable θv takes its value in the space Δv of conditional distributions on Xv given Xprt(v).
Let G† be the digraph with vertex set VG†=V∪ϑVa, where ϑVa={ϑv}v∈Va,
and arc set EG†=E∪{(ϑv,v),∀v∈Va}.
Such a graph G† is illustrated on Figure 6, where vertices in G†\G are represented as rectangles with rounded corners.
The augmented model of (G,ρ) is the collection of distributions factorizing on G† such that Xv is defined as in ρ for each v in V, Xθv=Δv, and
[TABLE]
where xprtG†(v)=(xprtG(v),θvo) for v∈Va, and xprtG†(v)=xprtG(v) for v∈Vs.
A distribution of the augmented model is specified by choosing the distributions of the θv.
In the rest of the paper, we denote by PG† the distribution of the augmented model with uniformly distributed θv for each v in Va.
With these definitions, a policy δ can now be interpreted as a value taken by θVa, and we have
[TABLE]
Note that in general PG†(XD=xD∣XM=xM) is the expected value over θVa of PθVa(XD=xD∣XM=xM).
The following result, which is an immediate consequence of (23), characterizes the pairs (D,M) such that the conditional distribution Pδ(XD∣XM) is the same regardless of the choice of policy δ.
Proposition 5**.**
We have Pδ(XD∣XM)=PG†(XD∣XM) for any PID on G, any policy δ, and any M such that Pδ(XM)>0 if and only if D is d-separated from ϑVa given M in G†.
Note that this is a particular case of a result known in the causality theory for graphical models (see *e.g., * Koller and Friedman, 2009, Proposition 21.3).
We have now all the tools to characterize C⊥⊥.
Theorem 6**.**
C⊥⊥* exists, is unique, and equal to
\Big{\{}v\in C\colon v\perp\vartheta_{V^{\mathrm{a}}}\,|\,C\backslash\{v\}\Big{\}}.*
With this characterization, the reader can check the value of C⊥⊥ on the example of Figure 5.
If we want to use the valid cuts in (21) in practice, we must to compute C⊥⊥ and p_{C^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}|C^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}} efficiently.
Theorem 6 ensures that C⊥⊥ is easy to compute using any d-separation algorithm (and more efficient algorithms are presumably possible),
and Proposition 5 ensures that, if we solve the inference problem on the RJT for an arbitrary policy, *e.g., *one where decisions are taken with uniform probability, we can deduce p_{C^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}|C^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}} from the distribution μC obtained.
Theorem 6 is an immediate corollary of the following Lemma, recently proved by two of the authors (Cohen and Parmentier, 2019, Theorem 1).
Lemma 7**.**
Let B and C be two sets of vertices. Then
M^{*}:=\Big{\{}v\in C\backslash B\colon v\,{\mathchoice{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\displaystyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\textstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 2.625pt\kern-4.11108pt\scriptstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 1.875pt\kern-3.3333pt\scriptscriptstyle\not\hss}{\perp}}}}\,B\,|\,C\backslash(B\cup\{v\})\Big{\}}
is a subset M of C such that
[TABLE]
Furthermore, if M satisfies (24), then M∗⊆M, so that M∗ is a minimum for the inclusion.
Cohen and Parmentier (2019) call M∗ the Markov Blanket of B in C. Note that if C=V this is the usual Markov Blanket.
Proof of Theorem 6.
Let M be a subset of C.
Proposition 5 ensures that Pδ(XC\M∣XM) does not depend on δ for any parametrization if and only if C\M⊥ϑVa∣M. Theorem 6 then follows by letting B=ϑVa in Lemma 7.
∎
Using the terminology of Cohen and Parmentier (2019), C^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}} is the Markov blanket of ϑVa in C.
5.3 Strength of the relaxations and their interpretation in terms of graph
Consider (G,ρ), a PID with G=(Vs,Va,E) and ρ=(X,p,r).
Let G be an RJT on G, and b an admissible bound satisfying (16).
The valid cuts of Section 5.1 enable to introduce the following strengthened version of the MILP (18).
[TABLE]
The following proposition summarizes the results of Section 5.1.
Proposition 8**.**
Any feasible solution (μ,δ) of the MILP (25)
is such that μ is the vector of moments of the distribution Pδ.
Hence, (μ,δ) is an optimal solution of (25) if and only if δ is an optimal solution of the MEU problem (3) on (G,ρ).
In this section we give interpretations of the linear relaxations of (18) and (25) in terms of graphs.
We introduce the sets of edges and IDs
[TABLE]
Figure 7 illustrates G and G⊥⊥ on the ID of Figure 2.
Note that E⊆E⊥⊥⊆E, and remark the three following facts on G and G⊥⊥.
First, the definition of both IDs depends on G and G.
Second, G is still an RJT on G and G⊥⊥.
And third, any parametrization (XV,p,r) of G is also a parametrization of G and of G⊥⊥.
The second and third results are satisfied by any ID G′=(Vs,Va,E∪E′), where E′ contains only arcs of the form (u,v) with v∈Va and u∈Cv.
Hence, if we denote by ΔG′ the set of feasible policies for (G′,XV,p,r), we can extend the definition of S(G) in Equation (13) to such G′
[TABLE]
Theorem 9**.**
We have
[TABLE]
and
[TABLE]
Hence, if (μ,δ) is a solution of the linear relaxation of (18), then δ is a policy on G, while if (μ,δ) is a solution of the linear relaxation of (25), then δ is a policy on G⊥⊥.
Remark furthermore that S(G′) is generally not a polytope.
Indeed, when G′=G, this is the reason why \eqrefpb:NLP is not a linear program.
An important result of the theorem is that S(G) and S(G⊥⊥) are polytopes, and MEU(G,ρ) and MEU(G⊥⊥,ρ) can therefore be solved using the linear programs maxμ∈Pv∈Vr∑⟨rv,μv⟩ and maxμ∈P⊥⊥v∈Vr∑⟨rv,μv⟩ respectively.
The proof of the theorem uses the following lemma.
Lemma 10**.**
Let v be a vertex in Va.
Then x_{C_{v}}\mapsto p_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}|C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}}(x_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}}|x_{C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash v},x_{v}) is a function of (x_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}},x_{C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash v}) only. Hence, if a distribution μCv satisfies \mu_{C_{v}}=\mu_{C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}}p_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}|C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}}, then C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}\perp v\mid C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash\{v\} according to μCv.
Proof.
Consider the augmented model PG†.
Let P be a Cv⊥⊥-v trail.
Let Q be the trail P followed by the arc (v,ϑv).
Given that v has no descendants in Cv (because of the hypothesis C˚v={v}), the vertex v is a v-structure of Q.
As v\in C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}, if P is active given C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash\{v\}, then P is active given C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}, which contradicts the definition of Cv⊥⊥. Hence, C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}\perp v\mid C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash\{v\} according to PG†, and x_{C_{v}}\mapsto p_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}|C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}}(x_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}}|x_{C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash v},x_{v}) is a function of (x_{C_{v}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}},x_{C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash v}) only.
The second part of the lemma is an immediate corollary.
∎
Proof of Theorem 9.
First, remark that,
once we have proved P=S(G) and P⊥⊥=S(G⊥⊥),
the result follows from Theorem 4.
We now prove P=S(G).
Let μ be in P.
Then μ is a vector of moments in the local polytope of the RJT G on G. Furthermore, since, first, for v∈Va, faG(v)=Cv , and second, for v∈Vs,μCv=μCˇvpv∣prtG(v) together with prtG(v)=prtG(v) imply that, according to μCv, Xv is independent from its non-descendants in G restricted to Cv given prtG(v) , Theorem 2 ensures that μ is a vector of moments of a distribution that factorizes on G, which yields P⊆S(G). Inclusion S(G)⊆P is immediate.
Consider now a vector of moments μ in P⊥⊥.
Given v∈Va, Lemma 10 and the definition of G⊥⊥ ensure that, according to μCv, variable Xv is independent from its non-descendants in G⊥⊥ in Cv, i.e., Cv⊥⊥\{v},
given its parents in G⊥⊥, i.e., C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\backslash v.
If v∈Vs, constraints μCv=μCˇvpv∣prt(v) still implies that Xv is independent from its non-descendants in Cv given its parents according to μCv, because by definition of G⊥⊥, for v∈Vs, we have prtG⊥⊥(v)=prtG(v). Theorem 2 again enables to conclude that P⊥⊥⊆S(G⊥⊥). Inclusion S(G⊥⊥)⊆P⊥⊥ is immediate.
∎
6 Soluble influence diagrams
In this section, we make the assumption that IDs are such that any vertex v∈V has a descendant in the set of utility vertices Vr, i.e., Vs∪Va=asc(Vr).
The following remark explains why we can make this assumption without loss of generality.
Remark 5*.*
Consider a parametrized ID (G,ρ) where G=(Vs,Va,E) and Vs is the union of chance vertices Vc and utility vertices Vr.
Let (G′,ρ′) be the ID obtained by removing any vertex that is not in Vr and has no descendant in Vr and restrict ρ accordingly. If a random vector XV factorizes as a directed graphical model on (V,E) and V′⊆V is such that asc(V′)=V′, then XV′ factorizes as a directed graphical model on the subgraph induced by V′ with the same conditional probabilities pv∣prt(v).
Hence, given a policy δ on (G,ρ) and its restriction δ′ to (G′,ρ′), we have \mathbb{E}_{\delta}\big{(}\sum_{v\in V^{\mathrm{r}}}r_{v}(X_{v})\big{)}=\mathbb{E}_{\delta^{\prime}}\big{(}\sum_{v\in V^{\mathrm{r}}}r_{v}(X_{v})\big{)} where the first expectation is taken in
(G,ρ) and the second in (G′,ρ′), and the two IDs model the same MEU problem.
∎
The proofs of this section are quite technical and can be found in Appendix B.
6.1 Linear program for soluble influence diagrams
Consider an ID G=(Vs,Va,E) with Vs being the union of chance vertices Vc and utility vertices Vr.
Given a policy (δu)u∈Va and a decision vertex v, we denote δ−v the partial policy (δu)u∈Va\v.
A policy (δv)v∈Va is called a local optimum if
[TABLE]
It is a global optimum if it is an optimal solution of (3).
Two concepts, s-reachability and the relevance graph have been introduced in the literature to characterize when a local minimum is also global (see *e.g., * Koller and Friedman, 2009, Chapter 23.5).
A decision vertex u is s-reachable from a decision vertex v if ϑu is not d-separated from dsc(v) given fa(v):
[TABLE]
The usual definition is \vartheta_{u}\,{\mathchoice{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\displaystyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\textstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 2.625pt\kern-4.11108pt\scriptstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 1.875pt\kern-3.3333pt\scriptscriptstyle\not\hss}{\perp}}}}_{G^{\dagger}}\,\mathrm{dsc}(v)\cap V^{\mathrm{r}}\mid\mathrm{fa}(v), but these definitions coincide in our setting, since we have assumed that dsc(v)∩Vr=∅ for any v∈Va.
Intuitively, the definition of this concept is motivated by the fact that the choice of policy δv given (δw)w=v depends on δu only if u is s-reachable from v. Note that, for example, if u∈dsc(v), then u is s-reachable from v.
The relevance graph of G is the digraph H with vertex set Va, and whose arcs are the pairs (v,u) of decision vertices such that u is s-reachable from v.
Finally, the single policy update algorithm (SPU) (Lauritzen and Nilsson, 2001) is the standard coordinate ascent heuristic for IDs. It iteratively improves a policy δ as follows: at each step, a vertex v is picked, and δv is replaced by an element in \displaystyle\operatorname*{\arg\!\max}_{\delta^{\prime}_{v}\in\Delta_{v}}\mathbb{E}_{\delta^{\prime}_{v},\delta_{-v}}\Big{(}\sum_{u\in V^{\mathrm{r}}}r_{u}(X_{u})\Big{)}.
The following proposition characterizes a subset of IDs, called soluble IDs, which are easily solved, and provides several equivalent criteria to identify them.
Proposition 11**.**
(Koller and Friedman, 2009, Theorem 23.5)*
Given an influence diagram G, the following statements are equivalent and define a soluble influence diagram.*
-
For any parametrization ρ of G, any local optimum is a global optimum.
2. 2.
For any parametrization ρ of G, SPU converges to a global optimum in a finite number of steps444In fact, if the graph is soluble, and if the decision nodes are ordered in reverse topological order for the relevance graph, then SPU converges after exactly one pass over the nodes.
.
3. 3.
The relevance graph is acyclic.
Given a parametrized influence diagram G and an RJT G, we introduced in Equation (13) the notation S(G) for the subset of the local polytope LG corresponding to moments of policies.
The following theorem introduces a new characterization of soluble IDs in terms of convexity.
Theorem 12**.**
If G is not soluble then there exists a parametrization ρ such that, for any junction tree G, the set of achievable moments S(G) is not convex.
If G is soluble, Algorithm 2 returns an RJT such that P⊥⊥=S(G) for any parametrization ρ.
The property of being soluble characterizes “easy” IDs that can be solved by SPU.
Theorems 4 and 12 imply that, if G is soluble, our MILP formulation 25 reduces to the linear program
[TABLE]
and is therefore “easy” to solve.
Of course, this property of being “easy” refers only to the decision part of the ID.
If a soluble ID is such that, given a policy, the inference problem is not tractable, both SPU and our MILP formulation will not be tractable in practice.
Theorem 12 is a corollary of Theorem 9 and the following lemma, and both results are proved in Section B.
Lemma 13**.**
There exists an RJT G such that G⊥⊥=G if and only if G is soluble.
Such an RJT can be computed using Algorithm 2.
Note that based on a topological order of the relevant graph, Algorithm 2 proceeds by computing a maximal perfect recall graph that contains graph G and that assigns the same parent sets to elements of Vs, then uses a topological order of this graph to order the nodes of G for the computation of a rooted junction tree.
6.2 Comparison of soluble and linear relaxations
MILP solvers are based on (much improved) branch-and-bound algorithms that use the linear relaxation to obtain bounds. Their ability to solve formulation (25) therefore depends on the quality of the bound provided by the linear relaxation.
As SPU solves efficiently soluble IDs, we could imagine alternative branch-and-bounds schemes that use bounds computed using SPU on “soluble graph relaxation” of influence diagrams.
We now formalize this notion and compare the two approaches.
A soluble graph relaxation of an ID G=(Vs,Va,E) is a soluble ID G′=(Vs,Va,E′) where E′ is the union of E and a set of arcs with head in Va.
Remark that Theorem 9 can be reinterpreted as the link between soluble graph relaxation and linear relaxations.
And since S(G)=P and S(G⊥⊥)=P⊥⊥, by Theorem 12, G⊥⊥ and G are soluble, and therefore soluble graph relaxations of G.
Since any feasible policy for the ID G is a feasible policy for a soluble graph relaxation G′, for any parametrization ρ, the value of MEU(G′,ρ),
which can be computed by SPU, provides a tractable bound on MEU(G,ρ).
Soluble relaxations can therefore be used in branch-and-bound schemes for IDs, as proposed in Khaled et al. (2013).
To compare the interest of such a scheme to our MILP approach we need to compare the quality of the soluble graph relaxation and linear relaxation bounds.
Let G′ be a soluble graph relaxation of G, applying Algorithm 2 on G′ provides an RJT such that E⊥⊥⊆E′
Indeed, by Lemma 13, v is d-separated from Cv\faG′(v) given prtG′(v) in G′, and therefore also in G, which implies E⊥⊥⊆E′. Thus, by Theorem 9,
the bound provided by the linear relaxation of the MILP (25) is at least as good as the soluble graph relaxation bound, and sometimes strictly better thanks to constraints (μ,δ)∈Qb.
7 Numerical experiments
In this section, we provide numerical experiments showcasing the results of the paper.
In particular, on two examples of varying size, we study the impact of the valid inequalities.
On such examples, we solve the MILP formulation (18) with improved McCormick bounds relying on Section D.3,
and valid inequalities from Section 5 obtained from the RJT of Algorithm 1.
More precisely we solve
max{ ∑v∈Vr⟨rv,μv⟩∣(μ,δ)∈Q,δ∈Δd}
where Q is one of the four following polytopes :
Q1=(P×Δ)∩Q1 (no cuts),
Qb=(P×Δ)∩Qb (McCormick only),
Q⊥⊥,1=(P⊥⊥×Δ)∩Q1 (independence cuts only),
Q⊥⊥,b=(P⊥⊥×Δ)∩Qb (McCormick and independence cuts).
The difficulty of an instance can be roughly measured by the number of feasible deterministic policies (Mauá et al., 2012b), i.e., \big{|}\Delta^{\mathrm{d}}\big{|}.
We have \big{|}\Delta^{\mathrm{d}}\big{|}=\prod_{v\in V^{\mathrm{a}}}\big{|}\mathcal{X}_{v}\big{|}^{\prod_{u\in\mathrm{prt}(v)}\big{|}\mathcal{X}_{u}\big{|}}. Therefore, the difficulty depends exponentially on \big{|}\mathcal{X}_{v}\big{|} for v∈fa(Va). In our examples, we assume that \omega_{a}=\big{|}\mathcal{X}_{v}\big{|} for all v∈fa(Va) and \omega_{s}=\big{|}\mathcal{X}_{v}\big{|} for all v∈V\fa(Va).
Each instance is generated by first choosing ωa and ωs.
We then draw uniformly on [0,1] the conditional probabilities pv∣prt(v) for all v∈V\Va and on [0,10] the rewards rv for all v∈Vr.
We repeat the process 10 times, and obtain therefore 10 instances of the same size.
The results are reported in Table 1.
The first column specifies the size of the problem, the second the approximate number of admissible strategies.
The third column indicates the cuts used.
In the last four columns, we report the integrity gap (*i.e., *the relative difference between the linear relaxation and the best integer solution), the final gap (relative difference between best integer solution and best lower bound), the improvement obtained over the solution given by SPU and the (shifted geometric mean of the) computation time for each instance.
All gaps are given in percentage. Computing times are given in seconds and correspond to the shifted geometric mean of the time over 10 instances. All values are averaged over the 10 instances.
In the last column, we write TL when the time limit is reached for the 10 instances of the same size.
Sometimes, the time limit is reached only for some of the 10 instances, and we end up with a non-zero average final gap together with an average computing time that is smaller than the time limit.
All mixed-integer linear programs have been written in Julia (Bezanson et al., 2017) with JuMP (Dunning et al., 2017) interface and solved using Gurobi 7.5.2.
Experiments have been run on a server with 192Gb of RAM and 32 cores at 3.30GHz.
For each program, we use a warm start solution obtained by running the SPU algorithm of Lauritzen and Nilsson (2001) on the instances.
For notational simplicity, and since it is unambiguous, in the rest of this section we use the same notation to refer to a given node of the graph and to refer to the random variable associated with this node.
7.1 Bob and Alice daily chess game
We consider the chess game example represented in Figure 2.
The beginning of the RJT built by Algorithm 1 for this example is represented in Figure 8
Since \vartheta_{a_{t-1}}\,{\mathchoice{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\displaystyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\textstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 2.625pt\kern-4.11108pt\scriptstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 1.875pt\kern-3.3333pt\scriptscriptstyle\not\hss}{\perp}}}}_{G^{\dagger}}\,\mathrm{dsc}(a_{t})\mid\mathrm{fa}(a_{t}) for all t∈[T], the chess game example is not a soluble ID, thus cannot be solved to optimality by SPU.
Table 1(a) reports results on the generated instances.
In this problem we see that we can tackle large problems : we can reach optimality in less than one hour for a strategy set of size 10144, and find a small provable gap on even bigger instances.
Moreover, we see that the independance cuts reduce the computation time by a factor 100, whereas the improved McCormick bounds yield less impactfull improvements.
However, on this problem the SPU heuristic yields good results that are marginally improved
by our MILP formulation. On this problem the main interest of our formulation is the bounds obtained.
In the next problem we show better improvement.
7.2 Partially Observed Markov Decision Process with limited memory
Another classical example of ID is the Partially Observed Markov Decision Process (POMDP) introduced in Section 1.
Figure 9 provides the graph representation of the POMDP with limited information. Since \vartheta_{a_{t-1}}\,{\mathchoice{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\displaystyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\textstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 2.625pt\kern-4.11108pt\scriptstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 1.875pt\kern-3.3333pt\scriptscriptstyle\not\hss}{\perp}}}}_{G^{\dagger}}\,\mathrm{dsc}(a_{t})|\mathrm{prt}(a_{t}) for all t∈[T], this ID is not soluble. Figure 10 represents the RJT built by Algorithm 1.
However, for v∈Va, Cv⊥⊥=∅ and the linear relaxation of Problem (18) does not enforce all the conditional independences that are entailed by the graph structure. Indeed, Theorem 9 ensures that the linear relaxation of the MILP (18) corresponds to solving Problem (3) on the graph G.
For this example G corresponds to the MDP relaxation, in which the decision maker knows the state st when he makes the decision at.
Therefore, the conditional independences st⊥at∣ot is no more satisfied.
Although we cannot enforce these independences with linear constraints, we propose slightly weaker independences: in particular,
we propose an extended formulation corresponding to the bigger RJT represented in Figure 10 to enforce for st to be conditionally independent from at given (st−1,at−1,ot) for t>1.
In such a RJT, we have C_{a_{t}}^{{\!\scriptscriptstyle\,\bot\!\!\!\bot\,\!}}=\big{\{}s_{t}\big{\}}.
Therefore, we can derive valid equalities (20) in P⊥⊥.
We demonstrate the efficiency of such inequalities by solving the different formulation on a set of instances, summed up in Table 1(b).
This example is harder to solve to optimality as we only reach strategy set of size 1072.
Further, we can see that there are some instances where SPU seems to reach a local maximum that
is improved by our MILP formulation. Once again the valid cuts significantly reduce the root linear
relaxation gap and the solving time, even on large instances.
Conclusion
This paper introduced linear and mixed integer linear programming approaches for the MEU problem on influence diagrams.
The variables of the programs correspond, for the distributions induced by feasible policies, to the collection of vector of moments of the distribution on subsets of the variables that are associated to nodes of a new kind of junction tree, that we call a rooted junction tree.
We have thus introduced as well algorithms to build rooted junction trees tailored to our linear and integer programs.
For soluble IDs, which are IDs whose MEU problem is easy, in the sense that it can be solved by the single policy update algorithm, we showed that it can also be solved exactly via our linear programs.
Furthermore, we characterized soluble IDs as the IDs for which there exists a rooted junction tree such that the set of possible vector of moments on the nodes of the tree is convex for any parametrization of the influence diagram.
Finally, we proposed a mixed integer linear programming approach to solve the MEU problem on non-soluble IDs, together with valid cuts.
The bound provided by the linear relaxation is better than the bound that could be obtained using SPU on a soluble relaxation.
Numerical experiments show that the bound is indeed better in practice.
Appendix A Rooted junction tree properties
In this section we present further technical results on RJT that are usefull in the analysis of our approach.
We start we generic properties of RJT.
Proposition 14**.**
Let G be an RJT on G.
-
If there is a path from u to v in G, then there is a path from Cu to Cv in G.
2. 2.
If dscG(u)∩dscG(v)=∅, then either there is a unique path from Cu to Cv or from Cv to Cu in G.
Proof.
Let G be an RJT on G.
Consider a vertex v of G and a node C of G containing v.
Since C is a node of Gv, and by definition of Cv, there exists a Cv-C path in G.
Now consider u∈prt(v). Since fa(v)⊆Cv, we have u∈Cv.
Thus there exists a Cu-Cv path in G. The first statement follows by induction along a u-v path in G.
We now show the second statement. Let w be a vertex in dscG(u)∩dscG(v),
then by the first statement there exists both a Cu-Cw and a Cv-Cw path in G.
As G is a rooted tree, this implies the existence of either a Cu-Cv path or of a Cv-Cu path in G.
∎
The following lemma is key in proving Theorem 2.
Lemma 15**.**
Let C,D be subsets of V such that fa(D)⊆C and dsc(D)∩C=D. Any distribution μC on C such that each v in D is independent from its non-descendants given its parents factorizes as
μC=μC\Dv∈D∏qv∣prt(v) where μC\D=xD∑μC and qv∣prt(v) is defined as ∑xC\prt(v)μC∑xC\fa(v)μC when the denominator is non-zero, and as [math] otherwise.
Proof.
Let ⪯ be a topological order on C such that u∈C\D and v∈D implies u⪯v.
Such a topological order exists since dsc(D)∩C=D. We have
[TABLE]
where the first equality is the chain rule and the second follows from the hypothesis of the lemma.
∎
Proof of Theorem 2.
Let G be an RJT on G.
Let C1,…,Cn be a topological ordering on G,
let C≤i=⋃j≤iCj,
and C<i=C≤i\Ci.
Let τ be a vector of moments satisfying the hypothesis of the theorem, and for each v in V, let
qv∣prt(v) be equal to ∑xC\prt(v)τCv∑xC\fa(v)τCv if the denominator is non-zero, and to [math] otherwise.
We show by induction on i that
[TABLE]
Suppose the result true for all j<i, with the convention that μ0=1.
We immediately deduce from the induction hypothesis that τCi′=∑xC≤i\Ci′μC≤i for all i′<i.
It only remains to prove to prove τCi=∑xC<iμC≤i.
By definition of an RJT, we have fa(Ci˚)⊆Ci.
Proposition 14 implies that dsc(Ci˚)∩Ci⊆Ci˚.
Indeed let u be in dsc(Ci˚)∩Ci. Then there is a Ci-Cu path as u∈dsc(Ci), and a Cu-Ci path as u∈Ci. Hence Cu=Ci and u∈Ci˚.
By Lemma 15, we have τCi=τCi\Ci˚∏v∈Ci˚qv∣prt(v). Let Cj be the parent of Ci in G, we have τCi\Ci˚=∑xCj\CiτCj=∑xC<i\CiμC<i, the first equality coming from the fact that (τC)C∈V belongs to the local polytope, and the second from the induction hypothesis.
Thus,
[TABLE]
which gives the induction hypothesis, and the theorem.
∎
Proof of Proposition 3.
Algorithm 1 obviously converges given that it has only a finite number of iterations. If G is not connected, the algorithm is equivalent to its separate application on each of the connected components, which each yield a tree. W.l.o.g., we prove properties of the algorithm under the assumption that G is connected.
To simplify notations, we denote Cv′ by Cv, and check that it indeed corresponds to the root node of v.
We first prove that ⪯ is a topological order on G. First, remark that (u∈Cv)⇒(u⪯v). Indeed, if u∈Cv, then either Step 4 of the algorithm ensures that u∈fa(v) and u⪯v or u∈/fa(v) and there exists x such that u∈Cx and Cv→Cx.
But by Step 6 of Algorithm 1, the fact that Cv→Cx entails that v is the maximal element of Cx\{x} for the topological order, so that u≺v.
Furthermore, Step 6 ensures that (Cu,Cv)∈A implies u∈Cv.
We deduce from the previous result that (Cu,Cv)∈A implies u⪯v, and ⪯ is a topological order on G.
Then we show that (9) holds. We first show that (u∈Cv)⇒Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv and u∈C′ for any C′ on path Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv. Either u=v and this is obvious, or u∈prtG(Cv); and by recursion either Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv or u∈Cr with Cr the root of the tree which is also the first element in the topological order. But, unless u=r, this is excluded given that u∈Cr implies u⪯r. Note that this shows that Cu is indeed the unique minimal element of the set {C:u∈C} for the partial order defined by the arcs of the tree.
To show the first part of (9), we just need to note that either u∈fa(v) and the result holds, or there must exist x such that Cv→Cx and u∈Cx and by recursion, there exists w such that Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw and u∈fa(w).
Finally, we prove that we have constructed an RJT. Indeed, if two vertices Cv and Cv′ contain u then
since G is singly connected, the trail connecting Cv and Cv′ must be composed of vertices on the paths Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto3.59995pt0.0pt\pgfsys@lineto13.6575pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm-1.00.00.0-1.03.59995pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCu and Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv′, and we have shown in the previous paragraph that that u belongs to any C′ on Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto3.59995pt0.0pt\pgfsys@lineto13.6575pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm-1.00.00.0-1.03.59995pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCu and Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv′, and so the running intersection property holds. Finally, property (ii) of Definition 1 must holds because the fact that Cu is minimal among all cluster vertices containing u together with the running intersection property entails that the cluster vertices containing u are indeed a subtree of G with root Cu.
∎
Proposition 3 provides a justification for Algorithm 1, but it characterizes the content of the cluster vertices based on the topology of the obtained RJT,
which is itself produced by the algorithm (note that the composition of cluster vertices depends only on ⪯ via the partial order of the tree). The cluster nodes of any RJT and those produced by Algorithm 1 admit however more technical characterizations using only ⪯ and the information in G, which we present next. These characterizations will be useful in Appendix B.
For each vertex v in V, let
[TABLE]
Proposition 16**.**
Let G=(V,A) be an RJT satisfying Cv˚={v}, and ⪯ be a topological order on G. Then ⪯ induces a topological order on G and
[TABLE]
Proof.
Let G=(V,A) be an RJT satisfying Cv˚={v}, and ⪯ be a topological order on G.
Property 1 in Proposition 14 ensures that ⪯ induces a topological order on G.
We start by proving (29a). Let v and w be vertices such that w≻v and that there is a v-w trail Q in V⪰v. Let s0,…,sk be the nodes where Q has a v-structure and t1,…,tk the nodes with diverging arcs in Q. Note that, since the trail is included in V⪰v, the first nodes of the trail have to be immediate descendants of v in G so that the trail takes the form v\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictures0\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto3.59995pt0.0pt\pgfsys@lineto13.6575pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm-1.00.00.0-1.03.59995pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpicturet1\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpicture s1…tk\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpicturesk\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto3.59995pt0.0pt\pgfsys@lineto13.6575pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm-1.00.00.0-1.03.59995pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpicturew, where possibly sk=w and the last arc is not present. Then, given that v≺s0, and that ⪯ is topological for G, Proposition 14.2 implies that Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCs0.
But by the same argument, Property 2 in Proposition 14 implies Ct1\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCs0, but since G is a tree and v≺t1, we must have Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCt1\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCs1. By induction on i, we have Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCsi and thus Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw, which shows Equation 29a.
We now prove (29d). Let u and v be two vertices such that u⪯v and there is a u-v trail P with P\{u}⊆V⪰v. Let w be the vertex right after u on P. We have u∈fa(w), w⪰v and there is a v-w trail in V⪰v, which implies Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw by (29a). But, since u⪯v, the u-v trail is also in V⪰u, which similarly shows that Cu\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCv. So by (8) we have proved (29d).
∎
Proposition 17**.**
*The graph G=(V,A) produced by Algorithm 1 is the unique RJT satisfying Cv˚={v} such that the topological order ⪯ on G taken as input of Algorithm 1 induces a topological order on G and the implications in (29) are equivalences.
*
Proof.
Note that some visual elements of the proof are given in Figure 11.
It is sufficient to prove the following inclusions
[TABLE]
Indeed, note that by Proposition 3, the obtained tree is an RJT so that, by Proposition 16, the reverse inequalities hold.
We prove the result by backward induction on (30b) and (30a). For a leaf Cv of G,dscG(Cv)={Cv} so that (30a) holds trivially
and Cv=fa(v) so that (30b) holds because u∈fa(v) implies u⪯v.
Then, assume the induction hypothesis holds for all children of a node Cv.
We first show (30a) for Cv , i.e. that (Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw)⇒(w∈T⪰v) (see Figure 11). Let Cx be the child of Cv on the path Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw. By Proposition 3,
we have v≺x, so that V⪰x⊂V⪰v.
Then, using the induction hypothesis, by (30b), (v∈Cx) implies that there is a v-x trail in V⪰v, and by (30a), Cx\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCw implies there is a trail x-w in V⪰x, so there is a v-w trail in v≺x in V⪰v, which shows the result.
We then show (30b) for Cv (see Figure 11). Indeed if u∈Cv, either u∈fa(v) and u is in the RHS of (30b), or there exists a child of Cv, say Cx such that u∈Cx and u≺v, because the algorithm imposes v=max⪯(Cx\{x}). Since Cv\leavevmodeto20.32pt\vboxto6.67pt\pgfpicture\makeatletter\lower-3.33301ptto0.0pt\pgfsys@beginscope\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@setlinewidth0.4pt\pgfsys@invoke \nullfontto0.0pt\pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.00.0pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.013.6575pt0.0pt\pgfsys@invoke \definecolorpgfstrokecolorrgb0,0,0\pgfsys@color@rgb@stroke000\pgfsys@invoke \pgfsys@color@rgb@fill000\pgfsys@invoke \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@beginscope\pgfsys@invoke \pgfsys@setdash3.0pt,3.0pt0.0pt\pgfsys@invoke \pgfsys@moveto0.0pt0.0pt\pgfsys@lineto10.05756pt0.0pt\pgfsys@stroke\pgfsys@invoke \pgfsys@beginscope\pgfsys@invoke \pgfsys@transformcm1.00.00.01.010.05756pt0.0pt\pgfsys@invoke \pgfsys@invoke \lxSVG@closescope\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope \pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\pgfsys@discardpath\pgfsys@invoke\lxSVG@closescope\pgfsys@endscope\hss\lxSVG@closescope\endpgfpictureCx there exists a path v-x in V⪰v, and using induction, by (30b), (u∈Cx) implies that
∃w such that u∈fa(w) and there exists a trail w-x in T⪰v. But we have shown in Proposition 3 that (v∈Cx)⇒(v⪯x), so T⪰x⊂T⪰v and we have shown that there exists a v-w trail in T⪰v with u⪯v and u∈fa(w), which shows the result.
∎
Appendix B Proofs of Section 6
For any set C and binary relation R, we denote CRv the set of vertices u in C such that uRv.
The following lemma will be useful in the proof of Lemma 13. Let H denote the relevance graph of G.
Lemma 18**.**
In general, u∈dsc(v)⇒(v,u)∈H. But, when G is soluble, then u∈dsc(v)⇔(v,u)∈H.
Proof.
Assume that u is s-reachable from v, that is (v,u) is an arc in H. We first show that this implies that u and v have descendants in common. Indeed, by definition of s-reachability, this means that there exist w∈dsc(v) and an active trail T from ϑu to w. Either, T is a directed path and w is also a descendant of u or T must have a v-structure. In the latter case, let x be the node with the v-structure closest to ϑu on T; since the trail is active, we must have x∈fa(v) but since x is a descendant of u, in that case, v must be a descendant of u. In both cases considered u and v have descendants in common.
Now, if u is not a descendant of v, then v is s-reachable from u, which is not possible as H is acyclic.
Hence u∈dsc(v).
∎
As an immediate consequence, we obtain the following corollary.
Corollary 19**.**
If G is soluble and ⪯ is a topological order on G, then its restriction ⪯H to Va is a topological order on the relevance graph H.
Proof of Lemma 13.
Let G be a soluble influence diagram.
We start by proving that Algorithm 2 with G as an input returns an RJT G.
It suffices to show that it is possible to compute topological orderings in
step 9, that is, to prove that H, G′ and G′′, defined in Algorithm 2, are acyclic.
H is acyclic because the ID is soluble.
We now prove that G′ is acyclic. As G is acyclic and by definition of G′, a cycle in G′ contains necessarily two vertices of Va. Let u and v thus be two distinct elements of Va.
Remark that, if there exists a path from u to v in G, then v is strategically reachable from u, and u⪯Hv.
Hence, by definition of G′, the indices of vertices in Va for ⪯H can only increase along a path in G′. There is therefore no cycle in G′ containing two vertices in Va, and thus no cycle in G′.
We now prove that G′′ is acyclic. Suppose that there is a cycle in G′′.
Let ⪯G′ be a topological order on G′, and let vh
be the
smallest vertex v for ⪯G′ in the cycle such that there is an arc (u,v) in E′′\E′ in the cycle.
And let (uh,vh) be the corresponding arc in the cycle.
Let (ul,vl) be the arc of E′ right before (uh,vh) in the cycle such that vl∈Va.
Arc (ul,vl) is possibly identical to (uh,vh).
By definition of G′, given two disjoint vertices u and v in Va, either (u,v)∈E′ or (v,u)∈E′.
Since vh⪯G′vl by definition of vh, we have either vh=vl or (vh,vl)∈E′.
And since all the arcs in the vl-uh subpath of the cycle are in E′, we have uh∈dscG′(vl).
Hence uh∈dscG′(vh), which contradicts the definition of E′′ in Step 7.
Hence, Algorithm 2 always returns an RJT, which we denote by G.
It remains to prove that G is such that C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\subseteq\mathrm{fa}(v) for each decision vertex v∈Va.
We start with two preliminary results.
Remark that E⊆E′ implies that ⪯ is a topological order on G.
Let ⪯H denotes its restriction to Va.
Corollary 19
ensures that ⪯H is a topological order on H.
Hence, we have
[TABLE]
Furthermore, if
u∈Va and v∈V⪰us, the definition of G′ implies the existence of a path from u to v in G, and hence the existence of a path from V⪰ua to v in G.
We now prove C_{v}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\subseteq\mathrm{fa}(v) for each v∈Va.
This part of the proof is illustrated on Figure 12.a.
Let v be a vertex in Va, let u∈Cv\fa(v), and let b∈V≺ua.
We only have to prove that u is d-separated from ϑb given fa(v).
We start by proving that u and v have common descendants.
Proposition 17 guarantees that (29d) is an equivalence.
Hence, there exists a u-v trail in V≻v.
Consider such a u-v trail Q with a minimum number of v-structures.
Suppose for a contradiction that Q has more than one v-structure.
Starting from v, let w1 be the first v-structure of Q and u1 bet its first vertex with diverging arcs u1.
Using the result at the end of the previous paragraph, we have u1∈dsc(V⪰va).
Since Q has been chosen with a minimal number of v-structures, we obtain u1∈dsc(V≻va).
Let a1 denote an ascendant of u1 in V≻va.
Since w1∈dsc(v), Equation (31) ensures that w1⊥ϑv∣fa(a1).
Hence, the v-wi path is not active given fa(a1), and it therefore necessarily intersects prt(ai). Hence, u1∈dsc(v), and Q there exists a u-v trail Q with fewer v-structures than Q, which gives a contradiction. Trail Q therefore has a unique v-structure, and u and v have a common descendant w.
Hence, if ϑb-u trail P is active given fa(v), then P followed by a u-w path is active given fa(v).
The fact that dsc(v)⊥ϑb∣fa(v) ensures that there is no-such path P, and we have proved that u is d-separated from ϑb given fa(v).
Conversely, let G be a non-soluble influence diagram, and G an RJT on G.
Let u and b be two vertices in Va such that \mathrm{dsc}(v){\mathchoice{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\displaystyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\textstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 2.625pt\kern-4.11108pt\scriptstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 1.875pt\kern-3.3333pt\scriptscriptstyle\not\hss}{\perp}}}}\vartheta_{u}|\mathrm{prt}(v) and \mathrm{dsc}(u){\mathchoice{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\displaystyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 3.75pt\kern-5.27776pt\textstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 2.625pt\kern-4.11108pt\scriptstyle\not\hss}{\perp}}}{\mathrel{\hbox to0.0pt{\kern 1.875pt\kern-3.3333pt\scriptscriptstyle\not\hss}{\perp}}}}\vartheta_{v}|\mathrm{prt}(u).
Without loss of generality, we assume that if there is a path between Cu and Cv, it is from Cv to Cu. To prove the converse, we prove that C_{u}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\neq\mathrm{fa}(u).
This part of the proof is illustrated on Figure 12.b.
There exists an active trail Q from w∈dsc(u) to ϑv given prt(u).
Starting from w, let x be the first vertex with diverging arcs of Q if Q contains such a structure, and be equal to v otherwise.
And let P be the w-x subtrail of Q.
Remark that P must be an x-w path in G, because any passing v-structure on P cannot be at a descendant of w, for it would then be a descendant of u which could not have any descendant in fa(u) as G is acyclic.
The path P contains no v-structure, and is active given fa(u). Hence, it does not intersect fa(u).
Since x and u have w as common descendant, Proposition 14 ensures that Cx and Cu are on the same branch of G.
If v=x, x∈asc(w) and there is a path in G from Cx to Cw, moreover, since we assumed Cu is a descendant of Cv in G, and since u∈asc(w), then the path from Cx to Cw contains Cu and all the vertices of P.
Now, if x=v, then x is the first vertex with diverging arcs, and in that case it belongs to asc(u), because Q∖P must contain at least one v-structure and any such v-structure can only be at a node in asc(u).
So, again, there is a path in G from Cx to Cw which contains Cu and all the vertices of P.
Starting from x, let y be the last vertex of P such that Cy is above Cu in G, and z be the child of y in P.
But since Q is active, the y-ϑv subtrail of Q is active given fa(u), and we therefore have C_{u}^{{\!\scriptscriptstyle\mathchoice{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\displaystyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 5.69441pt\kern-2.63887pt\textstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 2.30557pt\kern-2.05554pt\scriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}{\mathrel{\hbox to0.0pt{\kern 1.52779pt\kern-1.66664pt\scriptscriptstyle\not\hss}{\,\bot\!\!\!\bot\,}}}\!}}\neq\mathrm{fa}(u).
∎
Proof of Theorem 12.
If G is soluble, Lemma 13 ensures that Algorithm 2 builds an RJT G such that G⊥⊥=G, and Theorem 9 ensures that P⊥⊥=S(P).
Consider now the result for non-soluble IDs. Let G be a non-soluble influence diagram.
Let a and b be two decision vertices that are both strategically dependent on the other one.
First, we suppose that a∈/asc(b) and b∈/asc(a). Let P be a path from a to w∈dsc(b) with a minimum number of arcs, and Q be a b-w path with a minimum number of arcs. Then w is the unique vertex in the intersection of P and Q.
Let u and v be the parents of w in P and Q respectively.
Consider a parametrization where all the variables that are not in P or Q are unary, all the variables in P and Q are binary, all the variables in the a-u subpath of P are equal to Xa, all the variables in the b-v subpath of P are equal to Xb, and pw∣prt(w) is defined arbitrarily.
Let G be an arbitrary junction tree, C be its cluster containing fa(w).
Then choosing a distribution μa as policy δa and a distribution μb as policy δb implies that the restriction of μC to Xuv is μuv=μaμb. Hence, the marginalization on Xuv of the set of distributions μC that can be reached for different policy is the set of independent distributions, which is not convex. Hence, S(G) is not convex.
We now consider the case where a∈asc(b) or b∈asc(a).
W.l.o.g., we suppose a∈asc(b).
There exists a trail from ϑa to w in dsc(b) that is active given prt(b). Let Q be such a trail with a minimum number of v-structures.
And let P be a b-w path.
W.l.o.g., we suppose that w is the only vertex in both P and Q.
Let wb be the parent of w on P and ws0 its parent in Q.
Starting from w, let s0,…,sk−1 denote the vertices with divergent arcs in Q, let t1,…,tk the v-structures, and pℓ denote the parent of b that is below tℓ.
Finally let sℓ′ (resp. sℓ′′) denote the parent of tℓ (resp tℓ+1) on the sℓ-tℓ subpath (resp. sℓ-ti+1 subpath) of Q.
The structures that we have just exhibited entail that G contains a subgraph of the form represented on Figure 13.
Each dashed arrows correspond to a path whose length may be equal to [math], in which case the vertices connected by the path are the same.
We now introduce a game that we will be able to encode on the graph of Figure 13 and hence on G.
This game is a dice game with two players a and b.
Before rolling a uniform die with three faces, player a chooses to play 1 or 2, where “playing i” means observing if the die is equal to i, and passing this information to player b.
The die s0 is rolled.
If a has played 1 (resp. 2), he passes the information true to b if the die s0 is equal to 1 (resp. 2), and false if it is equal to 2 (resp. 1), or something else e.
Player b does not know what a has played.
Based on the information he receives, player b decides to play 1, 2, or joker, that we denote j.
If he plays j, then none of the player either earns or loses money. If he plays i in {1,2}, then both players earn i euros if die s0 is equal to i, and lose 10 euros otherwise. The goal is maximize the expected payoff.
This game has two locally optimal strategies δ1 and δ2. In strategy δi, player a plays i and b plays i if he receives true and j otherwise.
Both strategy are locally optimal: each players decision is the best possible given the other ones. But only strategy δ2 is globally optimal.
It changes nothing to the game if we add k−1 coin tosses Xs1,…,Xsk−1, and player b observes the k equality tests Xt1,…,Xtk−1, where Xtℓ=\mathbbm1(Xsℓ−1=Xsℓ).
Indeed, player b can compute ∑ℓ=1kxpℓ and knows that Xa=Xs0 if and only if this sum is even.
The parameterization of the influence diagram that enables to encode this game is specified on the right part of Figure 13.
For any x, the mapping \mathbbm1x(⋅) is the indicator function of x.
All the variables that are not on Figure 13 or on the paths on Figure 13 are unary.
All the variables along paths represented by dashed arrows are equal.
Policies δi can therefore be defined as
[TABLE]
where \mathbbm1i is the Dirac in i.
A technical case to handle is the one where a=a′=tk. In that case, we define Xa={0,1} and δai=\mathbbm1i(Xsk−1′′).
Consider now a junction tree G on G.
Let C be a node of G that contains fa(w). Then C contains both wb and ws0.
Let μC1 and μC2 be the distributions induced by δ1 and δ2 on XC, and μbs01 and μbs02 their marginalizations on Xwbws0.
Since Xwb=Xb and Xws0=Xs0,
μbs01 and μbs02 are the distributions induced by δ1 and δ2 on Xbs0.
Let μbs0=2μbs01+μbs02.
Denoting again \mathbbm1x the Dirac at x, we have
[TABLE]
We claim that there is no policy δ that induces distribution μbs0 on Xbs0.
Indeed, in a distribution induced by a policy δ, it follows from the parametrization that
if P(Xa=1)<1 and P(Xb=1)>0, then P(Xb=1,Xu=e)>0.
And, if P(Xa=2)<1 and P(Xb=2)>0, then P(Xb=2,Xu=e)>0.
(In both claims, “if P(Xa=1)<1” must be replaced by “if δa(xsk−1)=\mathbbm1i(xsk−1)” when a=tk).
As μub is such that P(Xb=1)>0, P(Xb=2)>0, and P(Xb=1,Xu=e)=0, it cannot be induced by a policy.
Hence, S(G) is not convex.
∎
Appendix C Algorithm to build a small RJT
In this section we present an algorithm to build a RJT without considering a topological ordering on the initial graph G=(V,A).
The only difference between Algorithms 1
and 3
is that the for loop along a reverse topological ordering of Algorithm 1 is replaced in Algorithm 3 by a breadth first search that computes online this reverse topological ordering.
Hence, if we denote ⪯ this ordering,
Algorithm 3 builds the same RJT as the one we obtain when we use Algorithm 1 with ⪯ in input.
Therefore, the RJT built by Algorithm 3 satisfies 9, and is such that the implications in (29) are equivalence.
Furthermore, Steps 5 and 6 enable to ensure that, when there is no path between a vertex u∈Va and a vertex v∈Vs, then u is placed before v in the reverse topological ordering computed by the breadth first search.
Therefore, ⪯ is a topological ordering on the graph G′′ used as Step 9 of Algorithm 2.
Hence, if G is soluble, Algorithm 3 builds a RJT such that G⊥⊥=G.
Remark that on non-soluble IDs, Steps 5 and 6 are a heuristic aimed at minimizing the size of Cv⊥⊥ for each v in Vs.
Such a heuristic is not relevant if valid cuts (20) are not used.
In that case, an alternative strategy could be to add as few variable as possible to Cv for v in Va to improve the quality of the soluble relaxation G.
This could be done by putting vertices u in Vs unrelated to v∈Va after in this topological order, i.e., by replacing Va by Vs in Steps 5 and 6.
Appendix D McCormick Relaxation
McCormick inequalities allow to turn the NLP formulation (11)
into the MILP formulation (18). Further good bounds ease the resolution
of Problem (18). In this section we first discuss these relaxation,
show that in the NLP formulation loose bounds are useless while tight bounds improve
the formulation. Finally, we give an algorithm to compute good quality bounds.
D.1 Review of McCormick’s relaxation
For the sake of completeness we briefly recall McCormick’s relaxation,
and condition for exactness if all of the variables but one are binary.
Proposition 20**.**
Consider the variables (x,y,z)∈[0,1]3 and the following constraint
[TABLE]
Further, assume that we have an upper bound y≤b.
We call McCormick\eqrefeq:prodcst the following set of contraints
[TABLE]
If x,y and z satisfy eq. 32, then they also satisfy eq. 33.
If x is a binary variable (that is x∈{0,1}) and eq. 33 is satisfied,
then so is eq. 32.
Proof.
Consider x∈[0,1], y∈[0,b] and z∈[0,1], such that z=xy.
Noting that (1−x)(b−y)≥0 we obtain Constraint (33a).
Constraints (33b) and (33c) are obtained
by upper bounding by bounding one variable.
Now assume that x∈{0,1}, y∈[0,b] and z∈[0,1] satisfy eq. 33.
Then, if x=1, constraints (33a) and (33b)
yield z=y. Otherwise, as z≥0, we have z=0 by (33c).
∎
D.2 Choice of the bounds in McCormick inequalities
D.2.1 Using bCˇv=1 leads to loose constraints
As μCˇv is a probability distribution, 1 is an immediate upper bound on μCˇv.
Let Q1 be the polytope Qb obtained using bounds vector b defined by bCˇv=1 for all v in Va.
Proposition 21**.**
Let μ be in P. Then there exists δ in Δ such that (μ,δ) belongs to Q1, and the linear relaxation of (18) is equal to μ∈Pmaxv∈Vr∑⟨rv,μv⟩.
Proof.
Let v be a vertex in Va, and let
[TABLE]
where ev is an arbitrary element of Xv.
To prove the result, we show that (McCormick(v,b)) is satisfied for this well-chosen δv∣prt(v) and bCˇv=1.
We have
[TABLE]
which yields μCv≥μCˇv+(δv∣prt(v)−1)bCˇv.
Besides, if μCv(xCv)≥0, following the definition of δ and given that μprt(v)(xprt(v))≤1, we have
[TABLE]
and the constraint μCv≤δv∣prt(v)bCˇv is satisfied.
Finally, μCv≤μCˇv follows from the marginalization constraint μCˇv=∑xvμCv in the definition of the local polytope.
∎
D.2.2 McCormick inequalities with well-chosen bounds are useful
This section provides examples of IDs where McCormick inequalities (McCormick(v,b)) improves the linear relaxation of MILPs (18) and (25).
Consider the ID on Figure 14.a,
and assume that we have a bound μst≤bst.
Then, the McCormick relaxation of μsta=μstδa∣t
reads
[TABLE]
Suppose that all variables are binary, that s is Bernoulli with parameter 21, that P(Xt=1∣Xs)=1+εXs−ε(1−Xs), that Xw indicates if Xs=Xa, and that the objective is to maximize Eδ(Xw), and has value 21+ε.
An optimal policy consists in choosing Xa=Xt.
An optimal solution of the linear relaxation of (18) on P without McCormick inequalities, has value 1.
Whereas an optimal solution with McCormick inequality and bst(xs,xt)=21+ε\mathbbm1xs=xt has value 21+ε.
However, on this simple example, the McCormick inequalities are implied by the valid inequalities of Section 5.
This is no more the case on the ID of Figure 14.b, where r is a Bernoulli of parameter 0.5 and Xs=XrXb+(1−Xr)(1−Xb), and the remaining of the parameters are defined as previously. Using the same bounds, this new example leads to exactly the same results as before.
D.3 Algorithm to compute good quality bounds
This section provides a dynamic programming equation to compute bounds bCˇv on μCˇv that are smaller than 1.
Let G be a RJT, and C1,…,Cn be a topological order on G. Let Ck be a vertex in G, Cj be the parent of Ck and Ci the parent of Cj (i<j<k). If k=1, then Ci=Cj=Ck=C1. We introduce the notation Cja=(Cj\(Ci∪Ck))∩Va.
We define inductively on k the functions b~k:XCk→[0,1] as follows.
[TABLE]
Proposition 22**.**
Let μ be in S(G).
We have μCk(xCk)≤b~k(xCk) for all i and xCk in XCk.
As a consequence, bCˇv defined as ∑xvb~Cv provides an upper bound on μCˇv that can be used in McCormick constraints.
Proof.
We prove the result by induction. Let (μ,δ) be a feasible solution of Problem 11 and Cj be the parent of Ck in G, and Ci the parent of Cj (i<j<k).
If k=1, then the result is obtained by using δv≤1 for all v∈Va.
We assume now that the induction is true until k>1. We have
[TABLE]
From (36) to (37), we maximize over the policies in (Ci∪Cj)\Ck. From (37) to (38), we bound all policies in Ck∩Va by 1. Then (39) is obtained by using the induction assumption. Let α:Xfa(Cja)→R be such that for all xCja∈XCja,
[TABLE]
Then, (39) becomes
[TABLE]
Now, we can suppose that Cv˚={v}. Therefore, ∣Cja∣≤1 and the maximum above can be decomposed into the sum.
[TABLE]
where from (41) to (42) we use a local maximization. Therefore, we obtain the result
[TABLE]
∎
Note that b~k(xCk) is computed via an order two recursion from b~i(xCi) where i is the grand-parent of k, which can be generalized to higher order if stricter bound are needed.
