Deception, Delay, and Detection of Strategies

TL;DR

This paper explores how strategies in complex systems can be concealed or revealed in stages, analyzing the structure of informative sequences that expose strategies in nondeterministic and stochastic graphs.

Contribution

It provides detailed proof and analysis of the existence and structure of informative action release sequences in strategy complexes of controllable graphs.

Findings

Maximal strategies contain at least one informative sequence of length at least n-1.

Number of such sequences is at least (n-1)! for each strategy.

Strategies can be hidden or revealed in stages, affecting detection and bluffing.

Abstract

Homology generators in a relation offer individuals the ability to delay identification, by guiding the order via which the individuals reveal their attributes (see arXiv:1712.04130). This perspective applies as well to the identification of goal-attaining strategies in systems with errorful control, since the strategy complex of a fully controllable nondeterministic or stochastic graph is homotopic to a sphere. Specifically, such a graph contains for each state $v$ a maximal strategy $\sigma_v$ that converges to state $v$ from all other states in the graph and whose identity may be shrouded in the following sense: One may reveal certain actions of $\sigma_v$ in a particular order so that the full strategy becomes known only after at least $n-1$ of these actions have been revealed, with none of the actions revealed definitively inferable from those previously revealed. Here $n$ is the number of states in the graph. Moreover, the strategy contains at least $(n-1)!$ such informative action release sequences, each of length at least $n-1$. The earlier work described above sketched a proof that every maximal strategy in a pure nondeterministic or pure stochastic graph contains at least one informative action release sequence of length at least $n-1$. The primary purpose of the current report is to fill in the details of that sketch. To build intuition, the report first discusses several simpler examples. These examples suggest an underlying structure for hiding capabilities or bluffing capabilities, as well as for detecting such deceit.

Equations

$$\begin{array}{cccccccccc}{\pi }_1:& \mathtt{H}& \to & \mathtt{L}& \to & \mathtt{P}& & & & \\ {\pi }_2:& \mathtt{H}& \to & \mathtt{R}& \to & \mathtt{P}& & & & \\ {\pi }_3:& \mathtt{H}& \to & \mathtt{L}& \to & \mathtt{R}& \to & \mathtt{P}& & \\ {\pi }_4:& \mathtt{H}& \to & \mathtt{R}& \to & \mathtt{L}& \to & \mathtt{P}& & \\ {\pi }_5:& \mathtt{H}& \to & \mathtt{L}& \to & \mathtt{R}& \to & \mathtt{L}& \to & \mathtt{P}\\ {\pi }_6:& \mathtt{H}& \to & \mathtt{R}& \to & \mathtt{L}& \to & \mathtt{R}& \to & \mathtt{P}\end{array}$$

$${\sigma }_{123}=\{0\to 1,0\to 2,0\to 3,1\to 4,2\to 5,3\to 6,4\to 7,5\to 7,6\to 7\}.$$

$${\sigma }_{23}=\{1\to 0,0\to 2,0\to 3,4\to 1,2\to 5,3\to 6,4\to 7,5\to 7,6\to 7\}.$$

$$\Phi_{R}=\bigcup_{x\in X}\hbox{$<$$

Y_{x}

$$>$}\qquad\hbox{and}\qquad\Psi_{R}=\bigcup_{y\in Y}\hbox{$<$$

X_{y}

$$>$}.$$

$${\varphi }_R(\sigma )$$

$${\psi }_R(\gamma )$$

$$y_1,y_2,\dots ,y_k,withk\ge 1,$$

$$y_i\notin ({\varphi }_R\circ {\psi }_R)(\{y_1,\dots ,y_{i-1}\}),for\; all\; 1\le i\le k.$$

$$R=\left\{(\gamma ,y)|y\in \gamma \in \mathfrak{M}\right\}.$$

$$\mathcal{K}=\left\{\kappa \subseteq \gamma \cup \{y\}|\kappa is\; a\; minimal\; nonface\; of\Gamma \right\}.$$

$$e_1,e_2,e_3$$

$$e_3,e_2,e_1$$

$$\begin{array}[]{llll}\sigma_{1},\sigma_{2},\sigma_{4}&\quad\sigma_{2},\sigma_{1},\sigma_{4}&\quad\sigma_{1},\sigma_{2},\sigma_{14}&\quad\sigma_{2},\sigma_{1},\sigma_{14}\\ \sigma_{4},\sigma_{14},\sigma_{1}&\quad\sigma_{14},\sigma_{4},\sigma_{1}&\quad\sigma_{4},\sigma_{14},\sigma_{2}&\quad\sigma_{14},\sigma_{4},\sigma_{2}\\ \sigma_{1},\sigma_{14},\sigma_{4}&\quad\sigma_{14},\sigma_{1},\sigma_{4}&\quad\sigma_{1},\sigma_{14},\sigma_{2}&\quad\sigma_{14},\sigma_{1},\sigma_{2}\\ \sigma_{2},\sigma_{4},\sigma_{14}&\quad\sigma_{4},\sigma_{2},\sigma_{14}&\quad\sigma_{2},\sigma_{4},\sigma_{1}&\quad\sigma_{4},\sigma_{2},\sigma_{1}.\end{array}$$

$$e_3,b_4,e_2{b}_4,e_3,e_2{b}_4,e_2,e_3.$$

$$e_1,e_3,a_1,a_2{e}_1,e_3,a_2,a_1{e}_3,e_1,a_1,a_2{e}_3,e_1,a_2,a_1.$$

$$a_i\notin ({\varphi }_A\circ {\psi }_A)(\{a_1,\dots ,a_{i-1}\}),fori=1,\dots ,k.$$

$$\{a_1^{\prime },\dots ,a_{i-1}^{\prime }\}\cup {\tau }^{\prime }\in {\Delta }_{G/W}but\{a_1^{\prime },\dots ,a_i^{\prime }\}\cup {\tau }^{\prime }\notin {\Delta }_{G/W}.$$

$${\diamond }_1\stackrel{a_1^{\prime }}{\to }{\diamond }_2\stackrel{a_2^{\prime }}{\to }\cdots \stackrel{a_{k-1}^{\prime }}{\to }{\diamond }_k\stackrel{a_k^{\prime }}{\to }{\diamond }_1$$

$${\mathfrak{C}}_H=\{\begin{array}{rlrlr}\left\{c_t|t\in W\right\},& & if|W|>1& & (with{c}_{t}as\; defined\; above);\\ \varnothing ,& & otherwise.& & \end{array}$$

$$w=v_1\stackrel{a_1}{\to }{v}_2\stackrel{a_2}{\to }\cdots v_k\stackrel{a_k}{\to }{v}_{k+1}=v.$$

$${\mathcal{F}}_{\sigma }(W)=\left\{v\in V|w{\ge }_{\sigma }v,for\; somew\in W\right\}.$$

$$w\to t\to w_1\to \cdots \to w_k=w,withk\ge 1.$$

$${\kappa }^{(j+1)}={\kappa }^{(j)}\cup \{c_1,\dots ,c_{k-1}\}.$$

$$J_c={\mathcal{F}}_{\eta }(T).$$

$$W_{\diamond }=V_1\cup \cdots \cup V_{k}and{B}_{\diamond }=\{c_1,\dots ,c_k\}\cup {\mathfrak{A}}_1\cup \cdots \cup {\mathfrak{A}}_k.$$

$$\kappa^{(j+1)}\cap B_{\diamond}\;\,=\,\;\{c_{1},\ldots,c_{k-1}\}\;\,\cup\,\;\bigcup_{i=1}^{k}\big{(}\kappa^{(j)}\cap{\mathfrak{A}}_{i}\big{)}$$

$$|{\kappa }^{(j+1)}\cap B_{\diamond }|$$

$$|{\tau }_{\circ }|$$

$${\mathcal{K}}_c$$

$$J_c$$

$${\sigma }_c$$

$$\mathcal{A}$$

Full text

The earlier work described above sketched a proof that *every * maximal strategy in a *pure nondeterministic * or *pure stochastic * graph contains *at least one * informative action release sequence of length at least $n-1$. The primary purpose of the current report is to fill in the details of that sketch. To build intuition, the report first discusses several simpler examples. These examples suggest an underlying structure for hiding capabilities or bluffing capabilities, as well as for detecting such deceit.

1. 1 Introductory Examples

2. 1.1 Paths and Constituent Transitions

3. 1.2 Strategies and Underlying Capabilities

4. 1.3 Hidden State

5. 2 Review of Prior Work and Notation

6. 2.1 Graphs and Strategies

7. 2.2 Relations and Dowker Complexes

8. 2.3 Minimal Nonfaces

9. 2.4 Sample Graphs, Relations, and Informative Action Release Sequences

10. 2.4.1 A Directed Cycle Graph

11. 2.4.2 A Graph with a Subspace Cycle

12. 2.4.3 An Augmented Cycle Graph

13. 3 Basic Tools

14. 4 The Nondeterministic Setting

15. 4.1 Hierarchical Cyclic Graphs

16. 4.2 Core Cycle Actions, Leaf Covers, Disruptive Sets of Actions

17. 4.3 Cycle-Breaking Strategies

18. 4.4 Markings

19. 4.5 Forward Projections

20. 4.6 Quotienting until Disruption

21. 4.7 Alternate Development: Quotienting until Disruption

22. 4.8 Acyclic Dissection Sizes

23. 4.9 Informative Action Release Sequences for Maximal Strategies

24. 4.10 Examples for Pure Nondeterministic Graphs

25. 4.10.1 A Hierarchical Pure Nondeterministic Graph

26. 4.10.2 A Pure Nondeterministic Graph with Several Nondeterministic Actions

27. 4.10.3 A Directed Graph with Several Cycles, Represented Hierarchically

28. 4.10.4 A Directed Graph with a Disruptive but not Cycle-Breaking Strategy

29. 5 The Stochastic Setting

30. 5.1 Expanding Fully Controllable Subgraphs via Minimal Nonfaces

31. 5.2 Informative Action Release Sequences from Expansive Sets of Actions

32. 5.3 An Informative Action Release Sequence from a Quotient

33. 5.4 Examples for Pure Stochastic Graphs

34. 5.4.1 A Directed Graph with Several Cycles

35. 5.4.2 A Pure Stochastic Graph

36. 5.4.3 A Pure Stochastic Graph Highlighting Expansive Set Order

37. 6 Counterexamples for Mixed Graphs

38. 6.1 A Counterexample with a Large Goal Set

39. 6.2 A Counterexample with a Small Goal Set

40. 6.3 A Counterexample with a Small Goal Set and Nonequivalent Inferences

41. Acknowledgments

1 Introductory Examples

1.1 Paths and Constituent Transitions

Figure 1 shows four islands connected by bridges, as might be found in one of the great oceanic cities of the world. One of the bridges allows traffic in two directions, the others are one-way bridges. Of interest are the possible paths a bus of tourists or the motorcade of a prominent dignitary might take from the Hotel Island to the Palace Island, via one or two intermediary islands (the Left Island and/or the Right Island).

Since the bridge between Left Island and Right Island is bidirectional, there are infinitely many such paths, parameterized by the number of times the bus or motorcade cycles over the two-way bridge. For the purposes of this report, we will disallow such infinite cycling. One can imagine different restrictions. In this first example, we impose the restriction that a vehicle may traverse the two-way bridge at most once in each of its possible directions (perhaps for legal or monetary reasons). In a more general setting, we would disallow traversing any bridge in the same direction more than once. Later, in Section 1.2, we will discuss a different example with a different restriction that prevents infinite cycling.

We define a permissible path to be any path that a vehicle might take from Hotel Island to Palace Island, subject to the “no directional transition twice” restriction. The next page enumerates all permissible paths; there are six. For clarity, we abbreviate each island name to its first letter and give paths the names $\pi_{1},\ldots,\pi_{6}$. Throughout this subsection, we consider only these six paths, each of which starts at Hotel Island and ends at Palace Island.

[TABLE]

Figure 2 describes the islands and bridges of Figure 1 as a directed graph, and the six permissible paths as a relation. The relation has a row for each permissible path and a column for each directed edge in the graph, that is, for each directional transition across a bridge. Since a permissible path may traverse any bridge direction at most once, each permissible path defines a set of directed edges, modeling all directional bridge transitions in the path. The relation therefore contains a nonblank entry $\bullet$ for a given path $\pi_{i}$ and a given directed edge ${{\tt A}\rightarrow{\tt B}}\kern 0.80002pt$ if and only if path $\pi_{i}$ includes transition ${{\tt A}\rightarrow{\tt B}}$.

Identifying Paths from Transitions at Execution Time

Suppose an observer is watching a bus drive from the Hotel Island to the Palace Island. At what point during the trip can the observer identify uniquely the specific path followed by the bus, assuming the bus is traversing one of the permissible paths $\pi_{1},\pi_{2},\pi_{3},\pi_{4},\pi_{5}$, or $\pi_{6}\kern 0.80002pt$?

• •

Certainly, once the bus arrives at its destination, Palace Island, the observer can identify the path uniquely, since at that point the observer knows that he/she has seen the entire path.

• •

The observer cannot identify any path uniquely after observing only the first bridge transition. For instance, after observing transition ${{\tt H}\rightarrow{\tt L}}$, the possible paths consistent with this observation are $\pi_{1}$, $\pi_{3}$, and $\pi_{5}$. Similarly, after observing transition ${{\tt H}\rightarrow{\tt R}}$, the possible paths consistent with the observation are $\pi_{2}$, $\pi_{4}$, and $\pi_{6}$.

• •

Consider paths $\pi_{1}$ and path $\pi_{2}$, each of which consists of two transitions. By the previous point, the observer must see the entire path in order to identify either of these paths uniquely.

• •

The first two transitions of paths $\pi_{3}$ and $\pi_{5}$ are the same, namely ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt L}\rightarrow{\tt R}}$. Consequently, upon observing these transitions, the observer cannot identify a path uniquely; the path could be either $\pi_{3}$ or $\pi_{5}$. Path $\pi_{3}$ consists of three transitions. Thus, if the actual path is $\pi_{3}$, the observer must see the entire path before identifying the path as $\pi_{3}$. A similar argument holds for path $\pi_{4}$.

• •

Paths $\pi_{5}$ and $\pi_{6}$ contain four transitions. For each of these two paths, the observer only needs to see the first three transitions in order to identify the path; no other permissible path shares those same three transitions with the path being observed.

In summary: Paths $\pi_{1}$, $\pi_{2}$, $\pi_{3}$, and $\pi_{4}$ can only be identified uniquely after seeing all their transitions, assuming one observes transitions in consecutive order. Paths $\pi_{5}$ and $\pi_{6}$ can be identified uniquely after seeing the first three of their four transitions, again assuming one observes transitions in consecutive order.

Identifying Paths from Transitions in Arbitrary Order

Previously we assumed that the observer was observing consecutive motions of a bus. Suppose now that the observer merely learns of particular transitions made by the bus, *without * any explicit ordering in time. For instance, perhaps the observer is listening to stories told by tourists on the bus after their trip, from which the observer attempts to reconstruct the path taken. Or perhaps the observations are coming from many trips taken over the course of several days by a bus following a particular fixed bus route each day. Or perhaps the observer overhears the bus driver commenting on particular bridges he will encounter on his next trip, from which the observer is trying to predict the path yet to be taken.

We now ask:  What *set * of transitions allows an observer to identify a path uniquely?

• •

Recall that path $\pi_{1}$ consists of the set of transitions $\{{{\tt H}\rightarrow{\tt L}},\,{{\tt L}\rightarrow{\tt P}}\}$. Previously, when observing transitions in consecutive order, seeing both these transitions identified path $\pi_{1}$ uniquely. That is no longer true when transitions may be observed nonconsecutively. The reason is that path $\pi_{5}$ contains these same transitions, plus others. In fact, it is no longer possible to identify path $\pi_{1}$ uniquely. Similarly, it is no longer possible to identify path $\pi_{2}$ uniquely.

• •

If the observer learns that a path contains the transitions ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt R}\rightarrow{\tt P}}$, then the observer can infer that the path must also contain the transition ${{\tt L}\rightarrow{\tt R}}$ and must in fact be path $\pi_{3}$. Whereas previously an observer needed to see the entire path $\pi_{3}$ in order to identify it uniquely, now a pair of nonconsecutive transitions identifies the path. In effect, continuity of paths allows the observer to infer an unobserved transition. A similar argument holds for path $\pi_{4}$. Of course, if a story teller wishes to draw out identification of the path, he/she might simply talk about the sights seen during the bus ride in consecutive order, thus preventing such a leap of inference.

• •

If the observer learns that a path contains the transitions ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt R}\rightarrow{\tt L}}$, then the observer can actually infer two unobserved transitions, namely ${{\tt L}\rightarrow{\tt R}}$ and ${{\tt L}\rightarrow{\tt P}}$, thereby concluding that the path is $\pi_{5}$. A similar inference is possible for path $\pi_{6}$. The observer is in effect taking advantage both of path continuity and knowledge of the path’s destination. Again, a story teller could draw out identification of path $\pi_{5}$ slightly by reporting transitions in consecutive order.

Figure 3 encodes these conclusions geometrically, using simplicial complexes [11, 13, 6]. As in the relation of Figure 2, we now view each path as a set of directed edges. These sets constitute the generating simplices of the left simplicial complex shown in Figure 3. The vertices in this complex are the directed edges of the graph of Figure 2. Path $\pi_{1}$ generates a one-dimensional simplex (edge) in the complex. This simplex is a subset of the three-dimensional simplex (tetrahedron) generated by path $\pi_{5}$, modeling the earlier conclusion that one cannot identify path $\pi_{1}$ uniquely when observing transitions in arbitrary order. Observe that the set $\{{{\tt H}\rightarrow{\tt L}},\,{{\tt R}\rightarrow{\tt P}}\}$ is a free face***Simplicial complexes in this report are abstract, i.e., collections of sets and all their subsets. A simplex is a *free face * of an abstract simplicial complex if it is a proper subset of exactly one maximal simplex in the complex. in the complex and is not itself a path. This geometry models the inference and identification of path $\pi_{3}$ discussed previously. Similarly, the set $\{{{\tt H}\rightarrow{\tt L}},\,{{\tt R}\rightarrow{\tt L}}\}$ forms a free face in the complex, modeling the inferences and identification of path $\pi_{5}$ discussed above. (The set $\{{{\tt H}\rightarrow{\tt L}},\,{{\tt R}\rightarrow{\tt L}}\}$ is an undrawn “diagonal” of the tetrahedron labeled $\pi_{5}$.)

The right simplicial complex of Figure 3 contains the same information as the left complex, but in dual form. The duality is with respect to the relation of Figure 2. (Details of such “Dowker duality” are discussed further in [6].)

The vertices in the right complex are the permissible paths of the graph of Figure 2. The generating simplices of the complex are given by the columns of the relation of Figure 2. In other words, each generating simplex consists of all the paths that share a given directed edge. Thus the right complex tells us how to interpret observations of transitions as intersections of generating simplices. For instance, if we know that a path contains the transitions ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt L}\rightarrow{\tt P}}$, then we can intersect the triangle labeled with ${{\tt H}\rightarrow{\tt L}}$ and the triangle labeled with ${{\tt L}\rightarrow{\tt P}}$ to see that the possible paths are $\pi_{1}$ and $\pi_{5}$. (This geometric intersection is exactly the intersection of the two columns indexed by ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt L}\rightarrow{\tt P}}$ in the relation of Figure 2.)

Considering such intersections, our earlier observations plus some others are immediate:

• •

$\pi_{1}$ and $\pi_{2}$ are not uniquely identifiable.

• •

Observing ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt R}\rightarrow{\tt P}}$ identifies path $\pi_{3}$.

• •

Observing ${{\tt H}\rightarrow{\tt R}}$ and ${{\tt L}\rightarrow{\tt P}}$ identifies path $\pi_{4}$.

• •

Observing ${{\tt H}\rightarrow{\tt L}}$ and ${{\tt R}\rightarrow{\tt L}}$ identifies path $\pi_{5}$ (as does observing ${{\tt L}\rightarrow{\tt P}}$ and ${{\tt L}\rightarrow{\tt R}}$).

• •

Observing ${{\tt H}\rightarrow{\tt R}}$ and ${{\tt L}\rightarrow{\tt R}}$ identifies path $\pi_{6}$ (as does observing ${{\tt R}\rightarrow{\tt P}}$ and ${{\tt R}\rightarrow{\tt L}}$).

(These are the smallest sets of identifying observations for each path; there exist larger sets of observations as well.)

1.2 Strategies and Underlying Capabilities

Figure 4 shows a river with two islands, a fishing area upstream of the islands, and a marina downstream from the islands. The two islands create three passages within the river that boats may traverse, either upstream or downstream, as they move between the fishing area and the marina. The three passages produce currents of different strengths. One of these currents is so strong that only boats with powerful motors are able to traverse the current going upstream. Fish in the fishing area like to gather near the start of that strong current. Consequently, boats with powerful motors have an advantage reaching nice fish over boats with weaker motors. On the other hand, revealing that one has a powerful motor leads to envy and other competitions. As a result, skippers tend to underplay the power of their motors.

We will examine the possible strategies for reaching the fishing area (along with strategies for reaching the marina). We will further examine the extent to which someone can reveal portions of a strategy without revealing the entire strategy. Conversely, we will examine the extent to which an observer can infer that a boat has a powerful motor even when the observer never sees the boat traversing upstream over the strong current.

Figure 5 describes the river setting of Figure 4 using a directed graph, much as we did in the earlier example of Section 1.1. We will be interested primarily in the transitions upstream and downstream through the passages beside the islands, so we give those directed edges explicit names: ${\tt u}_{1}$, ${\tt d}_{1}$, ${\tt u}_{2}$, ${\tt d}_{2}$, ${\tt u}_{3}$, ${\tt d}_{3}$, as shown in the right panel of the figure.

In the example of Section 1.1, we focused on paths. In the current example, we adopt a slightly different perspective. We are interested in a generalization of what is frequently called a control law, namely a mapping from states to commanded motions. The generalization is that of a strategy, to be reviewed in Section 2.1. A strategy is a mapping from states to *sets * of possible motions, in this case sets of directed edges. The semantics are as follows: When a boat is at a particular location, a strategy specifies a set of directed edges leading from that location to some neighboring locations. The boat must move along some one of those directions, with the particular direction determined possibly by circumstance rather than chosen by the skipper. (The strategy specifies a set since sometimes the precise direction is not so important as is a general direction. For instance, a boat with a powerful motor that is currently at the marina might be instructed to move toward any of the three passages in the river. A corresponding strategy would therefore include the set of transitions $\{0\rightarrow 1,\,0\rightarrow 2,\,0\rightarrow 3\}$.) If the set specified for a particular location is empty, then the boat must stop if it is at that location.

In the example of Section 1.1, we prevented infinite cycling by disallowing any path that traversed any directed edge more than once. With strategies, it is more natural to disallow any strategy whose motion sets might cause the system to revisit a state.

There is a well-developed theory for strategies in graphs with directed edges [1, 8, 9] as well as in graphs with nondeterministic and/or stochastic transitions [4, 5]. One can model the collection of all strategies as a simplicial complex, similar to the constructions of Section 1.1. In a directed graph, a strategy is a set of directed edges that produces no cycle(s) in the graph. A graph’s strategies constitute the simplices of a simplicial complex whose underlying vertex set consists of the graph’s directed edges. For a strongly connected directed graph, this simplicial complex has the homotopy type of a sphere, namely $\mathbb{S}^{\kern 0.56001pt{n-2}}$, with $n$ the number of states in the graph [8]. For the graph of Figure 5, the simplicial complex is therefore homotopic to $\mathbb{S}^{6}$.

Since a sphere has homology, our prior work on privacy [6] offers some lower bounds on how long a skipper may delay identification of a strategy or a boat’s final destination, relative to all possible strategies in the graph of Figure 5. However, rather than explore the entire space of strategies, we will focus in this example on some simpler scenarios.

Strategies for Attaining the Fishing Area

For the moment, let us consider only all maximal strategies that ultimately attain the fishing area from anywhere in the graph. (By a *maximal strategy * we mean here a cycle-free set of directed edges in the graph of Figure 5 that is maximal among all such sets.)

• •

Here is one such strategy, consisting of all possible upstream motions in the graph of Figure 5:

[TABLE]

(This strategy contains the three upstream transitions ${\tt u}_{1}$, ${\tt u}_{2}$, and ${\tt u}_{3}$, with ${\tt u}_{i}=i\rightarrow i+3$.)

• •

The strategy $\sigma_{123}$ only makes sense for a boat with a powerful enough motor to traverse the strong current of Figure 4. A boat without such a powerful motor might instead use the following strategy:

[TABLE]

(This strategy contains downstream transition ${\tt d}_{1}$ and upstream transitions ${\tt u}_{2}$ and ${\tt u}_{3}$.)

Strategy $\sigma_{23}$ is very similar to strategy $\sigma_{123}$, but in place of the upstream transitions $0\rightarrow 1$ and $1\rightarrow 4$, the strategy contains the downstream transitions $1\rightarrow 0$ and $4\rightarrow 1$. As a result, if necessary, the boat will first return to the marina via the leftmost passage of Figure 4, then move up to the fishing area via either of the other two passages.

Permissible Strategies:

Strategy $\sigma_{23}$ specifies two transitions at state #4, namely $4\rightarrow 7$ and $4\rightarrow 1$. A boat moving under strategy $\sigma_{23}$ may therefore reach the fishing area from state #4 either by moving directly to the fishing area or by moving first downstream to the marina then upstream via one of the other passages. Intuitively, this bifurcation arises because there are two arcs between any two points on a circle. Some maximal strategy must contain both.

While generally useful, motion multiplicity may merely add bookkeeping clutter, so we restrict it: To start, we define a *permissible strategy * to be a maximal strategy that (i) attains the fishing area from anywhere in the graph and (ii) specifies a unique motion at each state in the set $\{1,2,3\}$. We also stipulate that whenever a strategy specifies a passage transition and some other motion at a boat’s current location, then the boat will move through the passage.

We may now model permissible strategies via the relation of Figure 6. The relation contains a row for each permissible strategy and a column for each passage transition. An entry in the relation is nonblank if and only if the given strategy contains the given transition. Every maximal strategy in the graph of Figure 5 must contain exactly one transition from each of the three sets $\{{\tt u}_{i},{\tt d}_{i}\}$, $i=1,2,3$. Furthermore, among the  *permissible * strategies, each strategy is uniquely characterized by the three passage transitions it contains. (Of course, it is impossible for a permissible strategy to contain all three downstream transitions ${\tt d}_{1}$, ${\tt d}_{2}$, ${\tt d}_{3}$, since then the strategy would not be guaranteed to attain the fishing area from the marina.) Figure 6 further depicts a simplicial complex generated by the strategies of the relation. The underlying vertex set of this complex is $\{{\tt u}_{1},{\tt d}_{1},{\tt u}_{2},{\tt d}_{2},{\tt u}_{3},{\tt d}_{3}\}$, comprising the six passage transitions in the river.

Each of the free faces in the complex of Figure 6 consists of a pair of downstream transitions, e.g., $\{{\tt d}_{1},{\tt d}_{2}\}$, suggesting inference of an upstream transition, e.g., ${\tt u}_{3}$. Indeed, if an observer learns that a permissible strategy specifies downstream motion through two passages, then the observer can infer that the strategy must specify an upstream motion through the remaining passage (since the fishing area is given as destination). Consequently, observing two downstream transitions in a permissible strategy identifies the strategy uniquely. There are no other free faces in the complex. Consequently, observing any other proper subset of a permissible strategy’s passage transitions does not identify that strategy uniquely.

Comment:

On a given fishing expedition, an observer may only see a boat move through a single passage, but over the course of several days the observer may see the boat take different routes. Or perhaps a crewmember speaks of the transitions specified by a strategy. Assuming the skipper’s strategy is constant, the observer may be able to eventually infer the overall permissible strategy, much like an observer could infer a bus route in Section 1.1, after observing different bridge crossings on different days or by listening to tourist stories.

Inferring Motor Strength:

How might an observer of a boat’s transitions infer that the boat has a strong motor? Directly observing the upstream transition $u_{1}$ is one way, of course. Additionally, if the observer learns that a strategy specifies downstream transitions ${\tt d}_{2}$ and ${\tt d}_{3}$ *and * if the observer knows that these are part of a strategy to reach the fishing area, then the observer can infer that the strategy must contain the upstream transition ${\tt u}_{1}$, implying that the boat has a powerful motor.  (We assume that each boat only follows strategies it can execute.)

Four of the permissible strategies in Figure 6, namely $\sigma_{123}$, $\sigma_{12}$, $\sigma_{13}$, and $\sigma_{1}$, presuppose a powerful motor. If a skipper is following one of the strategies $\sigma_{123}$, $\sigma_{12}$, or $\sigma_{13}$, then the skipper can carefully reveal up to two different passage transitions while still hiding the motor’s power. In contrast, for strategy $\sigma_{1}$, the skipper can reveal at most one passage transition; revealing a second transition necessarily exposes or implies the motor’s power.

Strategies for Attaining the Fishing Area and Strategies

for Attaining the Marina

Let us augment our collection of permissible strategies, in order to model boat excursions that are *either * outbound to the fishing area  *or * returning to the marina. We now permit any maximal strategy for attaining the fishing area that specifies a unique motion at each state in the set $\{1,2,3\}$, plus  any maximal strategy for attaining the marina that specifies a unique motion at each state in the set $\{4,5,6\}$.  (We retain the stipulation regarding passage transitions.)

Focusing on the subcollection of strategies for attaining the marina, we may again construct a simplicial complex whose underlying vertex set is $\{{\tt u}_{1},{\tt d}_{1},{\tt u}_{2},{\tt d}_{2},{\tt u}_{3},{\tt d}_{3}\}$, much as in Figure 6, now with the upstream and downstream transitions interchanged. We thus have two simplicial complexes, one for the fishing-attaining strategies, the other for the marina-attaining strategies. We wish to combine these complexes. In order to not confuse simplices, we conify each complex with a vertex identifying the complex. We then glue the resulting two complexes together at common boundary locations. The final simplicial complex thus obtained is homotopic to $\mathbb{S}^{2}$.

In order to visualize this construction more easily, let us simplify the problem, by removing one of the islands, as in Figure 7. Now there are only two passages, one with a strong current requiring a powerful motor for the upstream direction, the other with a mild current, traversable by all boats in both directions. (The new graph’s state and transition names are consistent with those of Figure 5.  State #0 is the marina and state #7 is the fishing area.)

There are now three permissible strategies that attain the fishing area from anywhere in the graph. We name them $\sigma_{12}$, $\sigma_{1}$, and $\sigma_{2}$. Similarly, there are three permissible strategies that attain the marina from anywhere in the graph. We name them $\tau_{12}$, $\tau_{1}$, and $\tau_{2}$. Figure 8 describes these six strategies via a relation. Strategy names index the rows of the relation. The upstream and downstream transitions, ${\tt u}_{1}$, ${\tt d}_{1}$, ${\tt u}_{2}$, and ${\tt d}_{2}$, plus two additional attributes, ${\tt f}$ and ${\tt m}$, index the columns of the relation. Previously, when we were considering only permissible strategies for attaining the fishing area, the component upstream and downstream transitions of that strategy fully determined the strategy (given that the strategy was permissible). Now, knowing a strategy’s upstream and downstream transitions *may not * fully determine the strategy. However, each permissible strategy in the set $\{\sigma_{12},\sigma_{1},\sigma_{2},\tau_{12},\tau_{1},\tau_{2}\}$ is fully determined by its upstream and downstream transitions and by its destination. The attributes ${\tt f}$ and ${\tt m}$ model this destination. Attribute ${\tt f}$ means a strategy’s destination is the fishing area and attribute ${\tt m}$ means the strategy’s destination is the marina. For each permissible strategy, the relation lists the strategy’s upstream and downstream transitions along with its destination.

Comment:

One may readily observe a boat traversing a passage, but what does it mean to observe attribute ${\tt f}$ or ${\tt m}\kern 0.80002pt$?  One possibility is that a skipper announces a boat’s destination.  Another possibility is that the “observation” of ${\tt f}$ or ${\tt m}$ is actually an inference made from other observations. For instance, if an observer learns that the skipper has made preparations for both transitions $4\rightarrow 7$ and $5\rightarrow 7$, then the observer may conclude that the boat’s destination is the fishing area. Or perhaps someone observes a boat departing the marina, for instance by making the transition $0\rightarrow 2$. Then the observer knows that the boat’s destination cannot be the marina, and so has “observed ${\tt f}$” (assuming the strategy being observed is permissible).

We will now construct a simplicial complex to represent the relation of Figure 8, much as we constructed the complex in Figure 6. Let us proceed in steps. First, we construct a simplicial complex representing the permissible strategies for attaining the fishing area and a separate simplicial complex representing the permissible strategies for attaining the marina. In both cases, we let the underlying vertex set be $\{{\tt u}_{1},{\tt d}_{1},{\tt u}_{2},{\tt d}_{2},{\tt f},{\tt m}\}$. See Figure 9. Observe that attribute ${\tt f}$ is a cone apex for the complex generated by the fishing-attaining strategies, while attribute ${\tt m}$ is a cone apex for the complex generated by the marina-attaining strategies.†††A cone apex for a finite simplicial complex is a vertex contained in every maximal simplex of the complex. Finally, we glue these two simplicial complexes together along shared simplices, obtaining the complex of Figure 10. The homotopy type of this complex is $\mathbb{S}^{1}$, suggesting some ability to delay identification of strategies [6]. Observe in particular that every upstream or downstream transition appears in three permissible strategies.

Considering the free and nonfree faces of the simplicial complex in Figure 10, or directly from the relation of Figure 8, we may make the following inferences:

• •

Observing upstream transitions ${\tt u}_{1}$ and ${\tt u}_{2}$ in a boat’s strategy implies attribute ${\tt f}$. In other words, one may infer that the boat’s destination is the fishing area and that the encompassing permissible strategy is $\sigma_{12}$. This inference reflects the physical reality that a boat cannot have the marina as destination if its strategy always entails moving upstream, via the two passages on both sides of the island.

• •

Similarly, observation of ${\tt d}_{1}$ and ${\tt d}_{2}$ implies ${\tt m}$ and identifies strategy $\tau_{12}$.

• •

Observing an upstream transition in one passage and a downstream transition in the other passage (without knowing which occurred first) leaves the boat’s destination ambiguous. This ambiguity reflects the physical reality that the boat could have rounded either the upstream end or the downstream end of the island.

• •

Knowing the boat’s destination and observing a passage transition *away * from that destination implies the other passage transition and thus the overall permissible strategy. For instance, knowing that the boat is heading to the fishing area (${\tt f}$) and observing the boat move downstream along the strong current (${\tt d}_{1}$) means the boat’s strategy must also specify a motion upstream over the mild current (${\tt u}_{2}$). The geometry of the river forces this implication and thus identifies the strategy $\sigma_{2}$  (assuming strategies are permissible).

• •

Knowing the boat’s destination and observing a passage transition *toward * that destination leaves open the directionality of the transition through the other passage. For instance, knowing that the boat is heading to the marina (${\tt m}$) and observing the boat move downstream along the mild current (${\tt d}_{2}$) does not nail down whether the strategy specifies an upstream or a downstream transition through the passage with the strong current. There remains an ambiguity as to whether the encompassing permissible strategy is $\tau_{12}$ or $\tau_{2}$.

Deception and Detection

Skippers of boats with motors capable of moving upstream over the strong current tend to hide their strength for when it is really needed, such as a competition to snag nice fish at the head of the strong current. They may hide their strength either by never exercising it or by moving upstream over the strong current only under cover of fog or darkness.

Let us suppose that this deception is so pervasive that, for all intents and purposes, the upstream transition ${\tt u}_{1}$ is *never * observable. Two questions emerge:

How does the unobservability of ${\tt u}_{1}$ change the relation of Figure 8 and the complex of Figure 10? 2. 2.

How can one distinguish between a boat that is truly incapable of moving upstream over the strong current and a boat whose skipper is merely hiding that capability?

An initial answer to question #2 is that one cannot distinguish the two types of boats *if * the powerful boat *always * acts like the weaker boat. However, if the powerful boat does sometimes exercise its capabilities, then other observations *may * imply the boat’s power.

Figure 11 answers question #1. The relation one obtains when ${\tt u}_{1}$ is unobservable is the same as the original relation of Figure 8, except that the column indexed by the upstream transition ${\tt u}_{1}$ disappears. The resulting simplicial complex is now the deletion $\mathop{\rm dl}(\Gamma,{\tt u}_{1})$, with $\Gamma$ the complex of Figure 10. (Formally, $\mathop{\rm dl}(\Gamma,{\tt u}_{1})={\left\{\gamma\in\Gamma\;\left|\;{\tt u}_{1}\not\in\gamma\right.\right\}}$. In other words, the resulting complex contains all simplices of the original complex except those that included ${\tt u}_{1}$.)

In the new complex, some strategies that appeared as triangles originally now appear as edges. These are the strategies $\sigma_{12}$, $\sigma_{1}$, and $\tau_{2}$, namely all strategies that include the now unobservable upstream transition ${\tt u}_{1}$.

The observable portions of two of those strategies, namely $\sigma_{12}$ and $\tau_{2}$, are subsets of other strategies. For instance, the observable portion of $\sigma_{12}$ is a subset of strategy $\sigma_{2}$. This means: If one observes the upstream transition ${\tt u}_{2}$ and if one knows that the boat’s destination is the fishing area (attribute ${\tt f}$), then there remains an ambiguity regarding the strategy’s specified transition over the strong current; it could be either upstream (as in $\sigma_{12}$) or downstream (as in $\sigma_{2}$). Consequently, if a skipper with a powerful motor always follows strategy $\sigma_{12}$, but traverses the strong current only during fog and the mild current during clear weather, then no one will know of the boat’s power. (This assumes that the traversal times are unmeasured or constant, e.g., the skipper avoids traversing the mild current excessively quickly with the strong motor.)

In contrast, strategy $\sigma_{1}$ generates a maximal simplex even in the new complex and is thus uniquely identifiable from its observable attributes. This means: If one observes the downstream transition ${\tt d}_{2}$ but knows that the overall destination of the boat is the fishing area (attribute ${\tt f}$), then one can conclude that the strategy must be $\sigma_{1}$ and that the boat will traverse the strong current upstream. In other words, even though ${\tt u}_{1}$ is unobservable, one can infer its existence and conclude that the boat has a powerful motor.

It is instructive to construct the strategy relation and attendant simplicial complex for a boat that truly is incapable of traversing the strong current upstream. These appear in Figure 12. Not only does the transition ${\tt u}_{1}$ now disappear, but so do all strategies that relied on that transition. (Again, we assume that the boat only follows strategies which it is capable of executing.)

The space of strategies is much smaller now. The resulting simplicial complex looks similar to that in which ${\tt u}_{1}$ was merely unobservable, but there is a key difference: The simplex $\{{\tt f},{\tt d}_{2}\}$ is gone. (In fact, all three strategies that once contained ${\tt u}_{1}$ are now gone, but only one of those, namely $\sigma_{1}$, formed a maximal simplex previously in the complex of Figure 11.) The new complex tells us that it is *inconsistent * for a boat with a weak motor to announce its destination as the fishing area (attribute ${\tt f}$) but to traverse the mild current downstream (attribute ${\tt d}_{2}$).

Report [6] described how inconsistent observations sometimes suggest or identify unmodeled properties. Here, observing the inconsistency $\{{\tt f},{\tt d}_{2}\}$ might suggest that the actual strategies for the boat being observed are not those of Figure 12 but those of Figure 11.

We may therefore interpret the difference between the two relations and complexes of Figures 11 and 12 as follows:

• •

The relations and complexes tell an *actor * what behavior to avoid, in order to *be successful * at deception.

• •

The relations and complexes tell an *observer * how to  *look for * inconsistencies in behavior, in order to *detect * deception.

Inferences, Free Faces, Geometry

The simplicial complex of Figure 12 is a cone with apex ${\tt d}_{1}$. This geometry arises because *every * permissible strategy in the relation of that figure contains the downstream transition ${\tt d}_{1}$. Said differently, if a boat is incapable of moving upstream through the passage with the strong current, then there is no choice but to include the downstream transition in every permissible strategy for that boat. (Recall that a permissible strategy consists of a *maximal * set of motions that a boat *might * make to attain its destination without cycling.) One can thus infer the transition ${\tt d}_{1}$ “for free”, i.e., without observing or learning of the motion directly, assuming one knows that the boat has a weak motor.

Let us therefore remove vertex ${\tt d}_{1}$ from the complex of Figure 12 to obtain the following:

This simplified complex highlights some inferences that are possible when observing boats known to have weak motors (as usual, we also assume that the strategies under consideration are permissible strategies and that each boat only follows strategies it can execute):

• •

${\tt f}\!\implies\!{\tt u}_{2}$:  If one knows that a boat is heading to the fishing area, then one can infer that the boat must move upstream when traversing the mild current.

• •

${\tt d}_{2}\!\implies\!{\tt m}$:  If one observes a boat moving downstream over the mild current, then the boat must be heading to the marina.

These conclusions reflect the geometry of the fishing area and marina relative to the passages. Said differently, the geometry of the simplicial complex models the geometry of the river in such a way that one can draw conclusions about boat motions from the free faces in the complex. We saw this property as well when modeling paths in the example of Section 1.1.

1.3 Hidden State

The examples and discussion of this introduction have assumed that the underlying state is known to the observer. For example, the analysis of Section 1.2 assumed a state space defined by the river geometry, along with strategies whose motions were not allowed to revisit states.

In reality, a skipper might have additional hidden state. The restriction on revisiting states would then apply to the composite of observable and hidden states but not necessarily to the observable states alone. For instance, a skipper might be willing to revisit some parts of the river once or twice, relative to some hidden internal counter (perhaps fuel consumption). A skipper could then selectively hide some motions and reveal other motions, in order to bluff the capability of a strong motor. (An observer unaware of the skipper’s hidden state might infer ${\tt u}_{1}$ from observations $\{{\tt f},{\tt d}_{2}\}$ even though the boat only has a weak motor and has made surreptitious cyclic motions not involving ${\tt u}_{1}$ in order to give the appearance of a strong motor.)

One possible approach for dealing with such hidden state is to construct many possible models of that state, then hope to observe inconsistencies in behavior to rule out or imply some of these models. We leave such higher-order deception and detection for future work.

2 Review of Prior Work and Notation

This section briefly reviews key concepts and notation regarding strategy complexes and relations. Detailed discussions of strategies and strategy complexes appear in [4, 5]. The connection of strategies to relations appears in [6]. Background material on topology may be found in [11, 13], on posets in [15], and on privacy in [14, 2, 12, 3].

Assumption: All graphs, relations, and simplicial complexes in this report are finite.

2.1 Graphs and Strategies

This subsection reviews material on strategies, taken fairly directly from [5], with some descriptions verbatim.

Nondeterministic and Stochastic Graphs:

We are interested in finite graphs, viewed as state spaces with errorful transitions. We model any such graph $G$ as a pair $(V,{\mathfrak{A}})$, consisting of a finite set of *states * $\mskip 1.0muV\mskip-2.0mu$ and a finite collection of *actions * ${\mathfrak{A}}$. Each action $a\in{\mathfrak{A}}$ consists of a *source * state $v$ and a nonempty set $T\mskip-2.0mu$ of targets, with $v\in V$ and $\emptyset\neq T\subseteq V$.

If $\kern 1.19995pt{}T$ consists of a single state $t$, we say that the action is deterministic. We may write a deterministic action $a$ as $v\mskip-3.0mu\rightarrow\mskip-3.0mut$, just like a directed edge in a directed graph. If $\kern 1.19995pt{}T$ contains more than one state, there are two possibilities: The action is either *nondeterministic * or stochastic. We discuss each of these possibilities next. We may also regard a deterministic action as a special instance of a nondeterministic action and/or as a special instance of a stochastic action.

We write a nondeterministic action $a\,$ as $v\mskip-3.0mu\rightarrow\mskip-3.0muT$. The semantics of such an action are as follows: Action $a$ may be executed whenever the system is at state $v$. When action $a$ is executed, the system moves from state $v$ to one of the target states in $T$.  If $\,\lvert T\rvert>1$, then the precise target attained is not predictable in advance, but is known after execution completes. Different execution instances of action $a$ could attain different target states within $T$. An adversary might be choosing the target attained.

We write a stochastic action $a\,$ as $v\mskip-3.0mu\rightarrow\mskip-3.0mu\kern 0.80002pt{p}\,T$, with $p\kern 0.80002pt$ a strictly positive probability distribution $p:T\rightarrow(0,1]$, such that $\sum_{t\in{T}}p(t)=1$. The semantics of a stochastic action are very similar to those of a nondeterministic action, except that the target state attained at execution time is now determined stochastically, according to the probability distribution $p$, rather than nondeterministically. Different execution instances of action $a$ are assumed to be independent of each other.

We say that a graph $(V,{\mathfrak{A}})$ is a pure nondeterministic graph if all the actions in ${\mathfrak{A}}$ are either deterministic or nondeterministic. We say that a graph $(V,{\mathfrak{A}})$ is a *pure stochastic * graph if all the actions in ${\mathfrak{A}}$ are either deterministic or stochastic. This report is primarily interested in such pure graphs. However, see [5] for a discussion of graphs with a mix of deterministic, nondeterministic, and stochastic actions. See also Section 6.

Suppose action $a$ has source $v$ and targets $T$. We refer to each possible transition $\kern 0.80002pt{}v\rightarrow t\kern 0.80002pt$, with $t\in T$, as an action edge (of action $\kern 0.80002pt{}a$).

Comment:  We permit multiple actions to be distinct yet have the same source $v$ and the same target set $T$ (and the same probability distribution $p$ if the actions are stochastic). Such duplication flexibility is useful, for instance when forming quotient graphs (see page 2.1).

Strategies and Strategy Complexes:

We next define strategies and strategy complexes via a series of intermediate concepts. Intuitively, a strategy is a generalization of a control law, now viewed as a mapping from states to sets of actions.

Let $G=(V,{\mathfrak{A}})$ be a graph as on page 2.1, and suppose ${\mathcal{A}}\subseteq{\mathfrak{A}}$.  We view ${\mathcal{A}}$ as a generalized control law as follows: Suppose the system is currently at state $v$. The set ${\mathcal{A}}$ may contain zero, one, or several actions with source $v$. The system stops moving precisely when ${\mathcal{A}}$ contains no action with source $v$. Otherwise, the system must execute some action $a\in{\mathcal{A}}$ with source $v$. If there are several such actions, any one of the actions might execute, determined nondeterministically. (Worst-case, an adversary might make the choice. In Section 1.2, perhaps sometimes a boat’s skipper could.) Upon execution of action $a$, the system finds itself at one of the targets $t$ of action $a$. The process then repeats, with $t$ the system’s new current state.

We are interested in only those control laws that eventually stop at some state or states. Intuitively, for pure nondeterministic graphs, this means that executing any of the actions contained in ${\mathcal{A}}$ will never cause the system to cycle (i.e., revisit a previously encountered state).  For pure stochastic graphs, it means that no subset of the actions contained in ${\mathcal{A}}$ forms a recurrent Markov chain. We model these requirements with the following definitions, again taken fairly directly from [5]:

Let $G=(V,{\mathfrak{A}})$ be a graph as on page 2.1.

• •

With $a\in{\mathfrak{A}}$, let $\mathop{\rm src}(a)$ denote the source of action $a$. When ${\mathcal{A}}\subseteq{\mathfrak{A}}$, define ${\mathcal{A}}$’s *source set * $\mathop{\rm src}({\mathcal{A}})$ as the set of all the individual actions’ sources:  $\mathop{\rm src}({\mathcal{A}})={\left\{\mskip 1.0mu\mathop{\rm src}(a)\!\;\left|\;a\in{\mathcal{A}}\right.\right\}}$.

• •

With $a\in{\mathfrak{A}}$, let $\mathop{\rm trg}(a)$ denote the set of targets of action $a$.

• •

Let $W\subseteq V$ and $a\in{\mathfrak{A}}$. We say that action $a$ moves off  $\hbox{\hskip 0.2pt}{W}\!$ (in $G$) if $\mathop{\rm src}(a)\in W$ and one (or both) of the following is true: (i) action $a$ is nondeterministic with all of its targets in $V\setminus W$, or (ii) action $a$ is stochastic with at least one of its targets in $V\setminus W$. (The two requirements are identical when $a$ is deterministic.)

• •

Let ${\mathcal{A}}\subseteq{\mathfrak{A}}$. We say ${\mathcal{A}}$ *contains a circuit * if, for some nonempty subset ${\mathcal{B}}$ of ${\mathcal{A}}$, no action of ${\mathcal{B}}$ moves off $\mathop{\rm src}({\mathcal{B}})$. We say ${\mathcal{A}}$ *converges * or *is convergent * if ${\mathcal{A}}$ does not contain a circuit.   Comment:  If ${\mathcal{A}}$ is convergent and $V\!\neq\emptyset$, then necessarily $\mathop{\rm src}({\mathcal{A}})\neq V$.

• •

If $V\!\neq\emptyset$, then the strategy complex  $\Delta_{G}$ of  $G$ is the simplicial complex whose underlying vertex set is ${\mathfrak{A}}$ and whose simplices are all the convergent subsets ${\mathcal{A}}$ of ${\mathfrak{A}}$. Every simplex of $\Delta_{G}$ is called a strategy. The empty simplex $\emptyset$ is one such strategy, modeling no motion. It appears in $\Delta_{G}$ whenever $V\mskip-2.5mu\neq\emptyset$.  If $V\mskip-2.5mu=\emptyset$, then one would let $\Delta_{G}$ be the void complex, containing no simplices.  (This report will always require $V\!\neq\emptyset$.)

• •

If $\sigma$ is a strategy in $\Delta_{G}$, then $V\setminus\mathop{\rm src}(\sigma)$ is called the *goal * or *goal set * of $\sigma$. The goal set consists of all states in the graph at which $\sigma$ does not specify a motion. We may say that $\sigma$ *converges to * $V\setminus\mathop{\rm src}(\sigma)$.  If the goal set is a singleton $\{v\}$, we may refer directly to state $v$ as $\mskip 1.0mu\sigma$’s goal.

• •

Suppose $V\!\neq\emptyset$. We say that $G$ is *fully controllable * if every nonempty subset of $V\mskip-2.0mu$ is the goal set of some strategy in $\Delta_{G}$. Observe that $G$ is fully controllable if and only if every singleton state is the goal set of at least one *maximal * strategy (simplex) in $\Delta_{G}$.

• •

Topologically, $G$ is fully controllable if and only if $\Delta_{G}$ is homotopic to the sphere $\mathbb{S}^{\kern 0.56001pt{n-2}}$, with $n=\lvert V\rvert$.  See [4, 5].

We will soon model strategy complexes via relations. The maximal simplices of a strategy complex will index the rows of the relation and the graph’s actions will index the columns. Of interest will be how to reveal the constituent actions of a maximal strategy in such a way as to delay identification of the strategy for as long as possible.

We end this subsection with two definitions that will be useful later in the report:

Subgraphs:

Suppose $G=(V,{\mathfrak{A}})$ is a graph as on page 2.1.  By a subgraph $\mskip 1.0muH\mskip-1.5mu$ of $\mskip 4.0muG\mskip 2.0mu$ we mean a graph $H=(W,{\mathfrak{B}})\kern 0.80002pt$ in its own right such that $W\subseteq V$ and ${\mathfrak{B}}\subseteq{\mathfrak{A}}$.

Quotient graphs:

Suppose $G=(V,{\mathfrak{A}})$ is a graph as on page 2.1 and suppose $\emptyset\neq W\subseteq V$.

We define the *quotient graph * $G/W=(V^{\prime},{\mathfrak{A}}^{\prime})$ as follows:

• •

The state space is $\,V^{\prime}\;=\;(V\setminus W)\;\cup\;\{\diamond\}$.

Here $\diamond$ is a new state.  It represents the set of states $W\!$ all identified to one state.‡‡‡In this report, identify  typically means determine identity of, but sometimes, as here, it means  *treat as same *.

• •

The actions ${\mathfrak{A}}^{\prime}$ are in one-to-one correspondence with the actions ${\mathfrak{A}}$, but source and target states of actions in ${\mathfrak{A}}^{\prime}$ are relabeled to match the new state space. Specifically, any source or target in $V\setminus W$ remains unchanged, while any source or target in $W\mskip-0.5mu$ becomes $\diamond$.

The relabeling of targets may identify some or all of the targets of an action $a\in{\mathfrak{A}}$. For a stochastic action of the form $v\mskip-3.0mu\rightarrow\mskip-3.0mu\kern 0.80002pt{p}\,T$, with $T\cap W\mskip-2.0mu\neq\emptyset$, one therefore sums the transition probabilities of the targets in $T\cap W$ in order to determine the transition probability to state $\diamond$ of the relabeled action $a^{\prime}\in{\mathfrak{A}}^{\prime}$.

Comment: “one-to-one correspondence” means that distinct actions of ${\mathfrak{A}}$ remain distinct in ${\mathfrak{A}}^{\prime}$ even if their sources become the same and their target sets become the same (and even if their probability distributions become the same, in the stochastic case).

The following facts are easy to establish:

A convergent set of actions in $G$ may contain a circuit once one views the actions in $G/W$.  (In particular, individual actions may become nonconvergent.)  However, the set of actions remains convergent if its source set does not overlap $W$. 2. 2.

Any convergent set of actions in $G/W\mskip-1.5mu$ will remain convergent if one views the actions back in their original form in $G$. 3. 3.

If $G$ is fully controllable, then so is $G/W$.

Generalization:

Suppose $W_{1},\ldots,W_{k}$ are nonempty pairwise disjoint subsets of $V$, with $k\geq 1$.  Let $\diamond_{1},\ldots,\diamond_{k}$ be new and distinct states. The definition of quotient graph given above generalizes to this setting: For each $i=1,\ldots,k$, we identify all states of $G$ that lie in $W_{i}$ to the single state $\diamond_{i}$.  We denote the resulting quotient graph by $G/\{W_{1},\ldots,W_{k}\}$.

2.2 Relations and Dowker Complexes

This subsection reviews material on relations, taken fairly directly from [6], with some descriptions verbatim.

Let $X$ and $Y$ be nonempty finite discrete spaces. A relation $R$ on ${X}\mskip-1.8mu\times{Y}$ is a set of ordered pairs constituting a subset of the cross product ${X}\mskip-1.8mu\times{Y}$. We frequently view $R$ as a matrix of blank and nonblank entries, with $X$ indexing rows and $Y\mskip-1.0mu$ indexing columns. We often refer to elements of $\hbox{\hskip 1.0pt}{}X\mskip-0.5mu$ as  individuals and to elements of $\hbox{\hskip 1.0pt}{}Y\mskip-2.3mu$ as attributes.

For each $x\in{X}$, we let $Y_{x}$ be the set of attributes that individual $x$ has. Formally, $Y_{x}={\left\{\mskip 2.0muy\in Y\;\left|\;(x,y)\in R\right.\right\}}$. We may view $Y_{x}$ as the row of $R\mskip 0.75mu$ indexed by $x$ (or more precisely, as all the attributes with nonblank entries in the row indexed by $x$). Similarly, for each $y\in{Y}$, we let $X_{y}$ be the set of individuals who have attribute $y$, that is, $X_{y}={\left\{\mskip 1.0mux\in X\;\left|\;(x,y)\in R\right.\right\}}$. We may view $X_{y}$ as the column of $R\mskip 0.5mu$ indexed by $y$ (or more precisely, as all the individuals with nonblank entries in the column indexed by $y$).

Given a relation $R$, we define two simplicial complexes, $\Phi_{R}$ and $\Psi_{R}$, as follows:

$\Phi_{R}$ is called the Dowker attribute complex. It has underlying vertex set $Y\!$ and is generated by the rows of $R$.  $\Psi_{R}$ is called the Dowker association complex. It has underlying vertex set $X$ and is generated by the columns of $R$.  Thus:

[TABLE]

(The symbol <$$\sigma$$> means the simplicial complex generated by $\sigma$, that is, the collection of all subsets of $\sigma$, including the empty simplex $\mskip 2.0mu\emptyset$ and $\sigma$ itself.)

We define two interpretation maps, $\kern 0.80002pt\phi_{R}$ and $\psi_{R}$, as follows:

[TABLE]

Thus $\phi_{R}(\sigma)$ consists of all attributes shared by at least all the individuals in $\sigma$, while $\psi_{R}(\gamma)$ consists of all individuals who each have at least all the attributes in $\gamma$.

Observe that $\phi_{R}(\emptyset)=Y$ and $\psi_{R}(\emptyset)=X$.

One may regard $\phi_{R}$ both as an interpretation map as well as a test for membership in the Dowker complex $\Psi_{R}$. Specifically, for all $\sigma\subseteq{X}$, $\hbox{\hskip 1.0pt}\sigma\in\Psi_{R}\hbox{\hskip 1.0pt}$ if and only if $\hbox{\hskip 1.0pt}\phi_{R}(\sigma)\neq\emptyset$. Moreover, if $\hbox{\hskip 1.0pt}\emptyset\neq\sigma\in\Psi_{R}$, then $\emptyset\neq\phi_{R}(\sigma)\in\Phi_{R}$.

Similarly, for all $\gamma\subseteq{Y}$, $\hbox{\hskip 1.0pt}\gamma\in\Phi_{R}\hbox{\hskip 1.0pt}$ if and only if $\hbox{\hskip 1.0pt}\psi_{R}(\gamma)\neq\emptyset$. And $\emptyset\neq\psi_{R}(\gamma)\in\Psi_{R}$ whenever $\emptyset\neq\gamma\in\Phi_{R}$.

We say that individual $x\in X$ is identifiable via $\mskip 0.5muR\mskip 1.5mu$ whenever $\psi_{R}(Y_{x})=\{x\}$. In other words, an individual $x$ is identifiable when no other individual’s attributes include all of $x$’s attributes.

The following facts are useful to remember [6]:

Each of $\phi_{R}$ and $\psi_{R}$ is inclusion-reversing. 2. 2.

For all $\gamma\subseteq{Y}$,  $\gamma\subseteq(\phi_{R}\circ\psi_{R})(\gamma)$.  Similarly for $\psi_{R}\circ\phi_{R}$. 3. 3.

If $\gamma$ is a maximal simplex of $\Phi_{R}$, then $(\phi_{R}\circ\psi_{R})(\gamma)=\gamma$.  Similarly for $\psi_{R}\circ\phi_{R}$. 4. 4.

Each of the compositions $\phi_{R}\circ\psi_{R}$ and $\psi_{R}\circ\phi_{R}$ is idempotent. 5. 5.

$\phi_{R}\circ\psi_{R}\circ\phi_{R}=\phi_{R}$.  Similarly with the roles of $\phi_{R}$ and $\psi_{R}$ interchanged.

The maps $\phi_{R}$ and $\psi_{R}$ define homotopy equivalences between the two Dowker complexes [6]. In particular, the compositions $\phi_{R}\circ\psi_{R}$ and $\psi_{R}\circ\phi_{R}$ are homotopy equivalent to the identity maps on their respective simplicial complexes. These equivalences allow one to construct a poset whose elements may be viewed as pairs of sets $(\sigma,\gamma)$ satisfying $\emptyset\neq\sigma=\psi_{R}(\gamma)\in\Psi_{R}$ and $\emptyset\neq\gamma=\phi_{R}(\sigma)\in\Phi_{R}$. This poset has an encompassing lattice structure and is amenable to topological analysis: When the poset has high-dimensional homology, one can be assured that it contains long chains. We will not need the details of that poset construction in this report. Instead, we jump directly to one additional definition that we will need:

Informative Attribute Release Sequences:

An informative attribute release sequence (for relation $R$), abbreviated as iars, is a nonempty set of attributes in $\hskip 0.7pt{Y}$ released in a particular sequential order

[TABLE]

satisfying

[TABLE]

In order to understand this last condition, recall from [6] that $(\phi_{R}\circ\psi_{R})(\gamma)\setminus\gamma$, with $\gamma\in\Phi_{R}$, is the set of all attributes inferable from $\gamma$. For instance, one may have directly observed attributes $\gamma$ for some unknown individual known to be modeled by relation $R$. Then the set of attributes $(\phi_{R}\circ\psi_{R})(\gamma)\setminus\gamma$ is inferable without direct observation. Thus the condition above requires that no attribute $y_{i}$ in the sequence be inferable from the attributes $y_{1},\ldots,y_{i-1}$ released before $y_{i}$. In particular, $y_{1}$ must not be inferable “for free”, i.e., without observation. (The cone apex ${\tt d}_{1}$ of the complex in Figure 12 on page 12 was inferable for free and thus would never appear in an informative attribute release sequence for the relation in that figure.)

Comment:  An informative attribute release sequence $y_{1},y_{2},\ldots,y_{k}$ might not form a simplex in $\Phi_{R}$, but any proper prefix of the sequence will. It may at first seem counterintuitive to have a nonsimplex be informative, but the inconsistency one obtains with the last attribute released may provide information in some relation containing $R$, as discussed in [6].

Of interest in some privacy settings is how long one can delay identifying an individual while revealing information: Given an individual $x$, how large can one make $k$ in defining an informative attribute release sequence $y_{1},y_{2},\ldots,y_{k}$ for which $\psi_{R}(\{y_{1},\ldots,y_{k}\})=\{x\}$? Topology offers lower bounds [6]. In this report, we will consider that question with strategies in place of individuals and actions in place of attributes. We will argue from first principles.

Relations from Complexes:

Suppose $\Gamma$ is a nonvoid simplicial complex with underlying vertex set $Y\!\neq\emptyset$. We can define a relation $R$ on $\kern 0.80002pt{\mathfrak{M}}\times{Y}$, with ${\mathfrak{M}}$ the maximal simplices of $\Gamma$:

[TABLE]

One readily sees that $\Phi_{R}=\Gamma$.   (See footnote §§§Void vs. Empty: The void complex  is the simplicial complex $\Gamma=\emptyset$, containing no simplices. In contrast, the empty complex  is the simplicial complex $\Gamma=\{\emptyset\}$, containing a single simplex, namely the empty simplex  $\emptyset$. One may construct $R\mskip 1.0mu$ from the empty complex, assuming $\mskip 1.0muY\mskip-4.0mu\neq\emptyset$. In that case, ${\mathfrak{M}}=\{\emptyset\}$, $R=\emptyset$, and $\Phi_{R}=\{\emptyset\}=\Gamma$. for a side comment.)

Observe that every maximal simplex of $\Gamma$, i.e., every $\gamma\in{\mathfrak{M}}$, is identifiable via relation $R$.

Action Relations:

Suppose $G=(V,{\mathfrak{A}})$ is a graph as on page 2.1, now with both $V\mskip-3.65mu\neq\emptyset$ and $\mskip 1.0mu{\mathfrak{A}}\neq\emptyset$.  We may substitute $\Delta_{G}$ for $\Gamma$ in the previous construction to obtain a relation on ${\mathfrak{M}}\times{\mathfrak{A}}$, with ${\mathfrak{M}}$ the maximal simplices of $\Delta_{G}$. We refer to such a relation as an action relation, or more specifically, as graph $G$’s action relation. In an action relation, maximal strategies¶¶¶The term *maximal strategy (in $\mskip 2.0muG\mskip-1.0mu$ or $\mskip 1.0mu\Delta_{G}$) * is synonymous with maximal simplex (in $\Delta_{G}$). play the role of individuals while actions play the role of attributes. The strategy relations of Section 1.2 were of this form, though there we were only considering subcomplexes of the full strategy complex.

In this context, informative *attribute * release sequences become informative *action * release sequences.  We may thus ask the question:

How many actions can one reveal informatively before one has identified a maximal strategy?

Caution:

The order via which actions are revealed in an informative action release sequence need not correspond to the order in which actions might be executed at runtime.

Terminology:

Let $G$ be a graph as on page 2.1.  We will make statements of the form “maximal strategy $\sigma$ in $G\mskip 1.5mu$ contains informative action release sequence $a_{1},a_{2},\ldots,a_{k}$”. This statement means that the following three conditions hold:

• (a)

$\sigma$ is a maximal simplex in $\Delta_{G}$.

• (b)

$\{a_{1},a_{2},\ldots,a_{k}\}\subseteq\sigma$.

• (c)

$a_{1},a_{2},\ldots,a_{k}\mskip 2.0mu$ is an informative attribute release sequence for $G$’s action relation.

In particular, the order of the actions $\mskip 1.0mua_{1},a_{2},\ldots,a_{k}\mskip 1.0mu$ is significant. If we view the same set of actions in a different order we obtain a different sequence. Consequently, a statement of the form “$\sigma$ contains $k!$ different informative action release sequences” has meaning even when $\lvert\sigma\rvert=k$.  (Caution: A permutation of an informative attribute release sequence need not itself be an informative attribute release sequence. See page 2.4.2.)

2.3 Minimal Nonfaces

Suppose $\Gamma$ is a simplicial complex with underlying vertex set $Y$. A minimal nonface of (or in) $\Gamma\hbox{\hskip 0.4pt}$ is a subset of $\kern 1.19995pt{}Y\mskip-1.0mu$ that is not itself a simplex in $\Gamma$ but all of whose proper subsets are simplices in $\Gamma$.

If we arrange the vertices of a minimal nonface in any order, we obtain an informative attribute release sequence. That fact is the content of our first lemma:

Lemma 1** (Minimal Nonfaces as Informative Attribute Release Sequences).**

Let $R$ be a relation on ${X}\mskip-1.8mu\times{Y}$, with both $\hbox{\hskip 1.0pt}{}X\!$ and $\hbox{\hskip 1.0pt}{}Y\!$ nonempty. Suppose $\kappa$ is a minimal nonface of $\kern 1.49994pt\Phi_{R}$. Then any ordering of the attributes in $\kappa$ is an informative attribute release sequence for $R$.

Proof.

Suppose the lemma fails for some minimal nonface $\kappa$ of $\Phi_{R}$. Necessarily, $\kappa\neq\emptyset$. Let $k=\lvert\kappa\rvert$. Then, for some ordering $y_{1},y_{2},\ldots,y_{k}$ of the attributes in $\kappa$, $y_{k}$ must be implied by $y_{1},\ldots,y_{k-1}$ (if $k=1$, this means $y_{1}$ is inferable “for free”). Since $\kappa$ is a minimal nonface of $\Phi_{R}$, $\{y_{1},\ldots,y_{k-1}\}$ is a simplex in $\Phi_{R}$. Thus $\{y_{1},\ldots,y_{k-1}\}\subseteq(\phi_{R}\circ\psi_{R})(\{y_{1},\ldots,y_{k-1}\})\in\Phi_{R}$. By supposition, $y_{k}\in(\phi_{R}\circ\psi_{R})(\{y_{1},\ldots,y_{k-1}\})$. Consequently, $\kappa\subseteq(\phi_{R}\circ\psi_{R})(\{y_{1},\ldots,y_{k-1}\})\in\Phi_{R}$, contradicting the assumption that $\kappa$ is a nonface of $\Phi_{R}$. ∎

Suppose all individuals in a relation $R$ are identifiable. Then all rows of $R$ are distinct and each row forms a maximal simplex in the attribute complex $\Phi_{R}$. Suppose an observer has observed attributes $\eta$ for some unknown individual $x$ known to be modeled by relation $R$. Even if $\eta$ is a proper subset of $Y_{x}$, it is possible that $\eta$ identifies $x$, meaning $\psi_{R}(\eta)=\{x\}$. In that case, $Y_{x}$ is the only maximal simplex containing $\eta$. Conversely, if the observed attributes $\eta$ do not identify individual $x$, then $\eta$ must be contained in some maximal simplex besides $Y_{x}$. Thus there exists an attribute $y$ that is not one of $x$’s attributes but that is consistent with all the observed attributes $\eta$ of $x$, meaning $\eta\cup\{y\}\in\Phi_{R}$. The following lemma characterizes this situation more generally for a simplicial complex, in terms of minimal nonfaces.

Lemma 2** (Minimal Nonfaces between a Maximal Simplex and a Separate Vertex).**

Suppose $\Gamma$ is a simplicial complex with underlying vertex set $\kern 0.80002pt{}Y$.  Let $\gamma$ be a maximal simplex of $\,\hbox{\hskip 0.2pt}\Gamma$ and let  $y\in Y\!$ such that $y\not\in\gamma$.  Define

[TABLE]

Suppose $\eta\subseteq\gamma$. Let $\eta^{\prime}=\eta\cup\{y\}$.

Then $\kern 0.80002pt\eta^{\prime}\in\Gamma$  if and only if  $\,\kappa\setminus\eta^{\prime}\neq\emptyset\kern 0.80002pt$ for every $\kappa\in{\mathcal{K}}$.

Comments:  (i) ${\mathcal{K}}\mskip-2.0mu\neq\emptyset$, since $\gamma$ is a maximal simplex in $\Gamma$ and $y\not\in\gamma$.  (ii) If $\{y\}\not\in\Gamma$, then ${\mathcal{K}}$ consists solely of $\{y\}$ and $\{y\}\setminus\eta^{\prime}=\emptyset$, no matter what $\eta$ is. Indeed, no $\eta^{\prime}$ can be in $\Gamma$. (iii) If $\{y\}\in\Gamma$, then $\gamma$ cannot be the empty simplex. Every $\kappa\in{\mathcal{K}}$ now contains at least two vertices, namely $y$ and some element of $\gamma$. Therefore the lemma’s assertion for $\eta=\emptyset$ is clear. (iv) More generally, the lemma says: Even though vertex $y$ cannot enlarge simplex $\gamma$, it may be able to enlarge a face $\eta$ of $\gamma$. Such enlargement is possible precisely when the enlarged set contains no minimal nonfaces of the type described by ${\mathcal{K}}$.

Proof.

Suppose $\eta^{\prime}\in\Gamma$. If for some $\kappa\in{\mathcal{K}}$, $\kappa\setminus\eta^{\prime}=\emptyset$, then $\eta^{\prime}$ would contain $\kappa$ as a minimal nonface, a contradiction. Now suppose $\eta^{\prime}\not\in\Gamma$. Then $\eta^{\prime}$ must contain some minimal nonface, necessarily a set $\kappa$ in ${\mathcal{K}}$ since $\eta^{\prime}\subseteq\gamma\cup\{y\}$. Thus $\,\kappa\setminus\eta^{\prime}=\emptyset$. ∎

Minimal Nonfaces in a Strategy Complex:

Specializing to minimal nonfaces of a strategy complex yields additional results, as discussed below.

Lemma 3** (Minimal Nonfaces in Strategy Complexes).**

Let $G=(V,{\mathfrak{A}})$ be a graph as on page 2.1, with $\mskip 1.0muV\!\neq\emptyset$. Suppose $\kappa$ is a minimal nonface of $\kern 1.49994pt\Delta_{G}$. Then the actions in $\kappa$ all have distinct sources and no action in $\kappa$ moves off $\,\mathop{\rm src}(\kappa)$ in $\mskip 1.5muG$.

Proof.

Let $k=\lvert\kappa\rvert$.  Since $V\!\neq\emptyset$, $\,k>0$.

Write $\kappa=\{a_{1},\ldots,a_{k}\}$. For each $i=1,\ldots,k$, define $\kappa_{i}=\kappa\setminus\{a_{i}\}$, that is, remove one action from $\kappa$. Then $\kappa_{i}\in\Delta_{G}$, for $i=1,\ldots,k$. Thus, for every $\emptyset\neq\tau\subseteq\kappa_{i}$, some action in $\tau$ must move off $\mathop{\rm src}(\tau)$. On the other hand, $\kappa\not\in\Delta_{G}$, so for some $\emptyset\neq\xi\subseteq\kappa$, no action in $\xi$ moves off $\mathop{\rm src}(\xi)$. Consequently $\xi=\kappa$, establishing the second assertion of the lemma.

The first assertion is trivial if $k=1$, so assume $k>1$ and suppose $\mathop{\rm src}(a_{1})=\mathop{\rm src}(a_{2})$. Then $\mathop{\rm src}(\kappa)=\mathop{\rm src}(\kappa_{1})=\mathop{\rm src}(\kappa_{2})$. Some action in $\kappa_{1}$ moves off $\mathop{\rm src}(\kappa_{1})=\mathop{\rm src}(\kappa)$. That contradicts the previous paragraph, thereby establishing the first assertion of the lemma. ∎

Interpreting Minimal Nonfaces in Strategy Complexes:

Let us examine the meaning of minimal nonfaces for the two types of pure graphs discussed in this report. Assume the notation of Lemma 3 and its proof.

• •

Suppose $G$ is a pure nondeterministic graph. Inductively, Lemma 3 produces a cycle of actions $a_{1}$, …, $a_{k}$, such that $\mathop{\rm src}(a_{i+1})\in\mathop{\rm trg}(a_{i})$, for $i=1,\ldots,k$ (here indices wrap around, so that $a_{k+1}$ again means $a_{1}$). Moreover, for each action $a_{i}$, exactly one  of the action’s targets lies in $\mathop{\rm src}(\kappa)$; any additional targets lie outside $\mathop{\rm src}(\kappa)$. (Otherwise, one could create a shorter cycle and thus a proper subset of $\kappa$ would be a nonface of $\Delta_{G}$.)

• •

Suppose $G$ is a pure stochastic graph. In the definition of “moves off” from page • ‣ 2.1, the quantification over targets is different for stochastic actions than for nondeterministic actions. Consequently, Lemma 3 now implies that *all  * targets of every action in $\kappa$ must lie within $\mathop{\rm src}(\kappa)$. One may therefore create a subgraph $H$ of $G$ defined by $H=(\mathop{\rm src}(\kappa),\kappa)$. One sees that $\kappa$ is also a minimal nonface in $\Delta_{H}$, that $\Delta_{H}$ is the boundary complex∥∥∥The boundary complex on the set $Z$ (with $Z$ finite) is the simplicial complex whose underlying vertex set is $Z$ and whose simplices are all the proper subsets of $Z$. on the set $\kappa$, and that $H$ is a fully controllable pure stochastic graph. In fact, $H$ defines an irreducible Markov chain [7, 10, 5].

2.4 Sample Graphs, Relations, and Informative Action Release Sequences

This subsection provides examples of graphs, action relations, and strategy complexes, along with discussion of the extent to which strategy or goal identification may be delayed.  (Actions here are deterministic or nondeterministic. Stochastic actions appear in Sections 5 and 6.)

2.4.1 A Directed Cycle Graph

As a first example, consider the directed graph $G$ in the left panel of Figure 14. The graph contains four states and four directed edges. (The directed edges represent deterministic actions.) These edges form a directed cycle. Any proper subset of the four directed edges does not form a cycle. Consequently, any set of three directed edges forms a strategy, in fact a maximal strategy, that converges to one of the states in the graph, from any other state in the graph. For instance, the strategy $\sigma_{4}$, consisting of the set $\{e_{1},e_{2},e_{3}\}$ of directed edges, converges to state #4, for any initial starting state of the system.

Comment: Any subset of a maximal strategy is also a strategy, since it too will be acyclic. For instance, the set of directed edges $\{e_{1},e_{4}\}$ is a strategy that stops at either state #2 or state #3. (The precise stopping point depends on the starting point during a particular execution of the strategy.) The set $\{e_{1},e_{4}\}$ is a strategy but it it is *not * a *maximal * strategy.

The middle panel of Figure 14 shows graph $G$’s action relation, describing each maximal strategy by its constituent actions. For each state $v$, there is a maximal strategy $\sigma_{v}$ converging to that state from anywhere else in the graph. Therefore, $G$ is fully controllable. The strategy complex $\Delta_{G}$ is in fact generated by four such maximal strategies, each consisting of three directed edges. Consequently, the strategy complex is a hollow tetrahedron, as shown in the right panel of the figure. In particular, the strategy complex contains a single minimal nonface, namely the set $\{e_{1},e_{2},e_{3},e_{4}\}$, consisting of all four directed edges (actions) in the graph.

There are no free faces in the strategy complex, so it is impossible to infer any actions of a strategy from any actions revealed — “attribute privacy is preserved” [6]. Thus it is impossible to identify a maximal strategy uniquely if one knows only a proper subset of its actions. Each maximal strategy consists of three actions and has no free faces. Consequently, each maximal strategy contains $3!$ different informative action release sequences that identify the strategy, and each such sequence has length 3. For instance, the six sequences for strategy $\sigma_{4}$ are:

[TABLE]

Ability to Delay Strategy Identification:

Let $G=(V,{\mathfrak{A}})$ be a fully controllable graph with $n=\lvert V\rvert>1$. The following property holds [6]: For every state $v\in{V}$, there is some maximal strategy $\sigma_{v}\in\Delta_{G}$ such that $\sigma_{v}$ has goal $v$ and contains at least $(n-1)!$ different informative action release sequences of length at least $n-1$ each.  For each such sequence $a_{1},\ldots,a_{\ell}$, this means the following: An observer cannot infer (via $G$’s action relation) that $\sigma_{v}$ contains action $a_{i}$ merely from knowing that $\sigma_{v}$ contains the set $\{a_{1},\ldots,a_{i-1}\}$ of actions appearing earlier in the sequence. In particular, an observer cannot identify $\sigma_{v}$ uniquely before seeing all actions in the sequence $a_{1},\ldots,a_{\ell}$.  Moreover, $\ell\geq n-1$.

Comment: In the example of Figure 14, the six informative action release sequences of length 3 within each maximal strategy were permutations of the strategy’s three constituent actions. In general, the $\mskip 0.75mu(n-1)!\mskip 1.0mu$ different sequences need not be permutations of each other.

2.4.2 A Graph with a Subspace Cycle

As a second example, let us consider a graph with a directed cycle merely on a proper subset of the state space, as shown in Figure 15. The graph again consists of four states. The set of deterministic actions $\{e_{1},e_{2},e_{3}\}$ forms a directed cycle on the set of states $\{1,2,3\}$. In addition, there is a deterministic action $a_{2}$ that moves off this cycle space, specifically from state #2 to state #4. Finally, there is a nondeterministic action $b_{4}$ that moves from state #4 back to the set of target states $\{1,2,3\}$. (To say that the action is nondeterministic means that the precise target state attained cannot be predicted in advance, not even stochastically.)

The actions $e_{1}$, $e_{2}$, and $e_{3}$ form a directed cycle. Any two of these actions form a convergent strategy. The remaining two actions, $a_{2}$ and $b_{4}$, taken together, could cause the system to cycle between states #2 and #4. Any one of these actions is convergent by itself. Therefore, the two sets of actions $\{e_{1},e_{2},e_{3}\}$ and $\{a_{2},b_{4}\}$ each form a minimal nonface in the strategy complex $\Delta_{G}$. In fact, these are the only minimal nonfaces in the strategy complex. They are independent of each other. Consequently, $G$’s strategy complex is the simplicial join of the boundary of the triangle $\{e_{1},e_{2},e_{3}\}$ and the boundary of the edge $\{a_{2},b_{4}\}$. In other words, $\Delta_{G}$ is a suspension [11, 15] of a triangle boundary. Figure 16 depicts this complex along with $G$’s action relation. Observe that the complex is homotopic to $\mathbb{S}^{2}$, consistent with $G$ being fully controllable.

As in the example of Section 2.4.1, $\Delta_{G}$ contains no free faces. So, again, it is impossible to identify a maximal strategy uniquely from a proper subset of its constituent actions.

One salient difference between this example and the previous one is that some maximal strategies now have goal sets with more than one state in them. For instance, strategy $\sigma_{34}$, consisting of actions $\{e_{1},e_{2},a_{2}\}$ has goal set $\{3,4\}$. This multi-state goal arises because the actions $e_{1}$ and $e_{2}$, taken together, converge to state #3 *assuming * the system state lies within the subset of states $\{1,2,3\}$. However, if the starting state happens to be state #4, then the system will simply remain at that state. Consequently, the strategy $\{e_{1},e_{2}\}$ has goal set $\{3,4\}$. That strategy is not itself maximal. One can augment it either with action $b_{4}$, in which case the resulting maximal strategy would be $\sigma_{3}$, converging to state #3. Or one can augment $\{e_{1},e_{2}\}$ with action $a_{2}$ to produce $\sigma_{34}$. Adding action $a_{2}$ introduces some nondeterminism at state #2, but nothing that changes the overall goal set; it remains $\{3,4\}$.

Informative Action Release Sequences:

As in the example of Section 2.4.1, the longest informative action release sequences within each maximal strategy in $\Delta_{G}$ are simply permutations of the strategy’s constituent actions. Whenever a maximal strategy has no free faces, one can release its actions in any order without definitively identifying the strategy before all actions have been released.

In order to understand the more general picture, suppose we simply interchange the roles of strategies and actions in this example. The “individuals” are now $e_{1}$, $e_{2}$, $e_{3}$, $a_{2}$, $b_{4}$ and the “attributes” are $\sigma_{1}$, $\sigma_{2}$, $\sigma_{3}$, $\sigma_{4}$, $\sigma_{14}$, $\sigma_{34}$. We are thus interested in the Dowker association complex $\Psi_{A}$ of the original action relation. That complex appears in Figure 17.  (Comment: We are not asserting that this complex is the strategy complex of a fully controllable graph, merely using the complex to illustrate a point. One can however construct fully controllable graphs with strategy complexes that make the same underlying point: permutations of informative attribute release sequences need not themselves be informative attribute release sequences.)

Each maximal simplex of $\Psi_{A}$ continues to offer (at least) $3!$ different informative release sequences of length 3 each, with elements in each sequence drawn from the simplex’s vertices. Two of the maximal simplices (namely, the “endcaps” in the figure) are solid triangles with no free faces, so their sequences are again simply permutations of each other. Three of the maximal simplices are solid tetrahedra. The undrawn “diagonals” of these tetrahedra are free faces, so releasing their endpoints would completely identify the tetrahedron. For instance, releasing vertices $\sigma_{1}$ and $\sigma_{4}$ identifies the tetrahedron labeled $e_{3}$. Consequently, one cannot simply choose arbitrary sequences of length 3 and expect them to be informative. Nonetheless, each tetrahedron does contain 16 informative release sequences of length 3. Here are the sequences for the tetrahedron labeled $e_{3}$:

[TABLE]

Incorporating Additional Constraints:

Suppose, in some context, the system only executes strategies that converge to singleton goals. From an inference perspective, the action relation $A$ and strategy complex $\Delta_{G}$ of Figure 16 would be misleading. To understand the possible inferences, one should consider a relation $A^{(1)}$ that models all the maximal strategies with singleton goal sets, and only those, as shown in Figure 18.

With this added information, it is no longer true that one can find $6$ different informative action release sequences of length $3$ within every maximal strategy. For instance, action $a_{2}$ identifies strategy $\sigma_{4}$. Consequently, as soon as one releases that action, the other two actions in $\sigma_{4}$, if not previously released, would be implied. As a result, there are only two informative action release sequences of length 3 for identifying strategy $\sigma_{4}$, namely $e_{1},e_{3},a_{2}\mskip 1.0mu$ and $e_{3},e_{1},a_{2}$.

Similarly, action $e_{2}$ implies action $b_{4}$, again limiting the ordering of any sequences containing both those actions. Strategy $\sigma_{1}$ now contains only three, rather than six, informative action release sequences of length 3, namely:

[TABLE]

Ability to Delay Goal Identification:

Suppose $G=(V,{\mathfrak{A}})$ is a fully controllable graph with $n=\lvert V\rvert>1$. The following property holds [6]: For every state $v\in{V}$, there is some maximal strategy $\tau_{v}\in\Delta_{G}$ such that $\tau_{v}$ has goal $v$ and contains an informative action release sequence whose sequential release leaves the goal ambiguous at least until all actions in the sequence have been revealed.  Moreover, the sequence reduces the goal ambiguity by at most one state with each action revealed, so the sequence has length at least $n-1$.

One sees this property in the complex of Figure 18 since: (i) every maximal simplex contains a vertex shared by three strategies with different goals and (ii) the vertex lies within one of the simplex’s edges that is shared by two strategies with different goals.

2.4.3 An Augmented Cycle Graph

Let us augment the graph of Figure 14 with two nondeterministic actions, as shown in Figure 19. The actions are $a_{1}=1\mskip-3.0mu\rightarrow\mskip-3.0mu\{2,3\}$ and $a_{2}=2\mskip-3.0mu\rightarrow\mskip-3.0mu\{3,4\}$.

The new graph $\overline{G}$ has a strategy complex $\Delta_{\overline{G}}$ described by the action relation $\overline{A}$ shown in Figure 19. The complex is a partially puffed up version of the hollow tetrahedron from Figure 14, now consisting of three solid tetrahedra and two solid triangles glued together to enclose an $\mathbb{S}^{2}$ hole. Figure 20 shows the 1-skeleton of this complex.

The maximal strategies from the original strategy complex $\Delta_{G}$ are still present in $\Delta_{\overline{G}}$. Two of these strategies now lie within larger maximal simplices. For instance, strategy $\sigma_{2}$ for attaining goal state #2 in the original graph $G$ consisted of the actions $\{e_{1},e_{3},e_{4}\}$, whereas now the corresponding maximal simplex $\overline{\sigma}_{2}$ consists of the actions $\{e_{1},e_{3},e_{4},a_{1}\}$. The original strategy $\sigma_{2}$ always executed action $e_{1}$ when the system was at state #1, thus transitioning to state #3. In the new graph $\overline{G}$, the new $\overline{\sigma}_{2}$ might execute either action $e_{1}$ or action $a_{1}$ at state #1, selected nondeterministically (possibly by an adversary). Action $a_{1}$ will transition either to state #2 (the goal) or to state #3. (If an adversary controls the outcome of action $a_{1}$, then the adversary might choose to make action $a_{1}$ mimic action $e_{1}$, in which case the old $\sigma_{2}$ and the new $\overline{\sigma}_{2}$ would behave equivalently.)

Actions $e_{2}$ and $e_{4}$ both appear in both the original strategies $\sigma_{1}$ and $\sigma_{3}$ of $\Delta_{G}$. In the new graph $\overline{G}$, the set $\{e_{2},a_{1}\}$ contains a circuit, as does the set $\{e_{4},a_{2}\}$. Consequently, one cannot augment the strategies $\sigma_{1}$ and $\sigma_{3}$ with either of the actions $a_{1}$ or $a_{2}$. These strategies remain unchanged as one passes from $G$ to $\overline{G}$, that is, $\overline{\sigma}_{1}=\sigma_{1}$ and $\overline{\sigma}_{3}=\sigma_{3}$.

There are five maximal strategies in the new graph, whereas there were four previously. New strategy $\overline{\sigma}_{5}$ has the same goal state, namely state #4, as does strategy $\overline{\sigma}_{4}$, but arrives there with different actions, trading off action $e_{2}$ for action $a_{1}$. The minimal nonface $\{e_{2},\kern 0.80002pta_{1}\}$ of $\kern 0.80002pt\Delta_{\overline{G}}\kern 0.80002pt$ hints at this possible tradeoff.

The ability to delay strategy identification mentioned on page 2.4.1, as well as our analysis of the original graph $G$, ensures that each of $\overline{\sigma}_{1},\overline{\sigma}_{2},\overline{\sigma}_{3},\overline{\sigma}_{4}$ contains (at least) 6 informative action release sequences of length (at least) 3 each. What can we say about strategy $\overline{\sigma}_{5}$?

Releasing both of the two nondeterministic actions $a_{1}$ and $a_{2}$ identifies $\overline{\sigma}_{5}$. Releasing either one of these actions implies both deterministic actions in $\overline{\sigma}_{5}$. Consequently, one obtains the longest possible informative action release sequences within $\overline{\sigma}_{5}$ by first revealing the two deterministic actions (in either order) and then the two nondeterministic actions (in either order). Here are the four possible longest informative action release sequences within $\overline{\sigma}_{5}$:

[TABLE]

(The previous reasoning can be generalized and formalized using lattice representations of links, as discussed in [6], but we will not develop or use that machinery in this report.)

Informative Action Release Sequences for Maximal Strategies in Pure Graphs:

Suppose $G$ is a fully controllable graph containing $n$ states, with $n>1$. Suppose further that $G$ is either pure nondeterministic or pure stochastic. The remainder of this report will provide proofs that *every * maximal strategy in $G$ contains at least one informative action release sequence of length at least $n-1$. The proofs will be different for the two types of graphs. This property need not hold for graphs containing a mix of nondeterministic and stochastic actions. See [6] and Section 6 for counterexamples.

3 Basic Tools

Graphs and strategies were defined in Section 2.1.  We wish to prove the following theorem:

Theorem 4** (Informative Action Release Sequences for Maximal Strategies).**

Let $G=(V,{\mathfrak{A}})$ be a fully controllable graph with $n=\lvert V\rvert>1$. Suppose that $G$ is either pure nondeterministic or pure stochastic.  Then every maximal strategy in $\Delta_{G}$ contains an informative action release sequence of length at least  $n-1$.*

Section 4 will prove Theorem 4 for the pure nondeterministic case, while Section 5 will prove Theorem 4 for the pure stochastic case.

This section builds some tools that are useful for both settings. Throughout this section the graph $G$ may in fact contain a mix of deterministic, nondeterministic, and stochastic actions.

Terminology and Notation:

• •

We frequently abbreviate informative action release sequence as iars, for both the singular and plural cases.

• •

If $G$ is a graph, the phrase $a_{1},\ldots,a_{k}$* is an iars for $G$* means that $a_{1},\ldots,a_{k}$ is an informative action release sequence for $G$’s action relation as defined on page 2.2.

• •

Suppose $G=(V,{\mathfrak{A}})$ is a graph and $G^{\prime}=(V^{\prime},{\mathfrak{A}}^{\prime})$ is some quotient graph of $G$. Recall from page 2.1 that the sets of actions ${\mathfrak{A}}$ and ${\mathfrak{A}}^{\prime}$ are in one-to-one correspondence. Corresponding actions $a\in{\mathfrak{A}}$ and $a^{\prime}\in{\mathfrak{A}}^{\prime}$ differ only in that the source and/or target states of action $a^{\prime}$ may have changed from those of action $a$ in order to reflect the quotient graph’s state identifications. One may therefore view any action of $G^{\prime}$ in the form $a^{\prime}$, with $a$ being the unique action of $G$ corresponding to $a^{\prime}$.  We will use this prime notation from now on.

Our first tool allows us to combine two strategies when one of the strategies comes from a subgraph and the other comes from a quotient graph formed by collapsing that subgraph to a single state.

Lemma 5** (Combining Quotient and Subgraph Strategies).**

Let $G=(V,{\mathfrak{A}})$ be a graph and let $H=(W,{\mathfrak{B}})$ be a subgraph of $G$.  (As always, both $\hbox{\hskip 0.75pt}{}V\mskip-4.0mu$ and $\hbox{\hskip 0.5pt}{}W\mskip-4.0mu$ are assumed to be nonempty.)

If $\hbox{\hskip 0.4pt}\sigma^{\prime}\in\Delta_{G/W}$ and $\hbox{\hskip 0.2pt}\gamma\in\Delta_{H}$, then $\hbox{\hskip 0.2pt}\sigma\cup\gamma\in\Delta_{G}$.

(Prime notation indicates corresponding actions in a graph and a quotient, as discussed.)

Comment: Since all sources and targets of actions in $\gamma$ lie within $W$, no such action is convergent in the quotient graph $G/W$. Therefore $\sigma\cap\gamma=\emptyset$.

Proof.

Suppose $\emptyset\neq\tau\subseteq\sigma\cup\gamma$. We need to show that some action of $\tau$ moves off $\mathop{\rm src}(\tau)$ in $G$.

If $\mskip 1.0mu\mathop{\rm src}(\tau\!\setminus\!\gamma)\cap{W}=\emptyset$, then this assertion follows from Lemma 7.3(b)(i) in [5].

Otherwise, let $\kappa=\tau\!\setminus\!\gamma$. Then $\emptyset\neq\kappa^{\prime}\subseteq\sigma^{\prime}\in\Delta_{G/W}$, so some action $b^{\prime}$ in $\kappa^{\prime}$ moves off $\mathop{\rm src}(\kappa^{\prime})$ in $G/W$. If $b^{\prime}$ is nondeterministic, then all its targets lie in the set \big{(}(V\setminus W)\cup\{\diamond\}\big{)}\setminus\hbox{\hskip 0.2pt}\mathop{\rm src}(\kappa^{\prime}). If $b^{\prime}$ is stochastic, then at least one of its targets lies in that set. (As usual, $\mskip 0.5mu\diamond$ represents $\mskip 0.75mu{W}\mskip-1.5mu$ identified to a single state in $\mskip 1.0muG/W$.)  Since $\mskip 0.1mu\kappa\mskip 0.4mu$ contains an action with source in $W\mskip-1.5mu$, $\mskip 1.0mu\kappa^{\prime}$ contains an action with source $\diamond$.  Thus \big{(}(V\setminus W)\cup\{\diamond\}\big{)}\setminus\hbox{\hskip 0.2pt}\mathop{\rm src}(\kappa^{\prime})\;=\;V\setminus\big{(}W\cup\mskip 0.5mu\mathop{\rm src}(\kappa^{\prime})\big{)}\;=\;V\setminus\big{(}W\cup\mskip 1.0mu\mathop{\rm src}(\kappa)\big{)}\;\subseteq\;V\setminus\mathop{\rm src}(\tau). That means action $b$ lies in $\tau$ and moves off $\mathop{\rm src}(\tau)$ in $G$. ∎

The next lemma ensures that unquotienting a simplicial informative action release sequence (iars) again produces an informative action release sequence, when the quotienting is over a proper subspace that is fully controllable.

Lemma 6** (Lifting Quotient IARS).**

Let $G=(V,{\mathfrak{A}})$ be a graph and let $H=(W,{\mathfrak{B}})$ be a fully controllable subgraph of $G$, with  $\emptyset\neq W\subsetneq V$.

Suppose $a^{\prime}_{1},\ldots,a^{\prime}_{k}$ is an iars for $G/W$, with $\{a^{\prime}_{1},\ldots,a^{\prime}_{k}\}\in\Delta_{G/W}$ and $\kern 1.19995pt{}k\geq 1$.

Then $a_{1},\ldots,a_{k}$ is an iars for $G$.

Proof.

Let $A$ be $G$’s action relation. We need to show that

[TABLE]

Suppose this assertion is false for some $i$. Then $a_{i}$ is contained in every maximal simplex of $\Delta_{G}$ that contains $\{a_{1},\ldots,a_{i-1}\}$.

Let ${\mathfrak{A}}^{\prime}$ be the actions of the quotient graph $G/W$. Since $a^{\prime}_{1},\ldots,a^{\prime}_{k}$ is an iars for $\mskip 0.5muG/W\mskip-0.5mu$ and $\{a^{\prime}_{1},\ldots,a^{\prime}_{k}\}\in\Delta_{G/W}$, there exist actions $\tau^{\prime}$, with $\emptyset\neq\tau^{\prime}\subseteq{\mathfrak{A}}^{\prime}\setminus\{a^{\prime}_{1},\ldots,a^{\prime}_{i}\}$, such that

[TABLE]

Let $\eta^{\prime}$ be a minimal nonface of $\Delta_{G/W}$ contained in $\{a^{\prime}_{1},\ldots,a^{\prime}_{i}\}\cup\tau^{\prime}$. (Since $\{a^{\prime}_{i}\}\in\Delta_{G/W}$, $\eta^{\prime}$ contains at least two actions, one of them being $a^{\prime}_{i}$.)  By Lemma 3 on page 3, no action of $\eta^{\prime}$ moves off $\mathop{\rm src}(\eta^{\prime})$ in $G/W$.

We consider two cases below, deriving a contradiction for each.

By Lemma 3, there are no further cases.

State $\diamond$ represents $W\mskip-1.5mu$ identified to a single state in the quotient graph $G/W$, as per page 2.1.

• I:

No action in $\eta^{\prime}$ has source $\diamond$:

Then $\mathop{\rm src}(\eta)=\mathop{\rm src}(\eta^{\prime})$, so no action of $\eta$ moves off $\mathop{\rm src}(\eta)$ in $G$.

On the other hand, $\{a_{1},\ldots,a_{i-1}\}\cup\tau\in\Delta_{G}$, by Fact 2 on page 2.1.

So $\{a_{1},\ldots,a_{i}\}\cup\tau\in\Delta_{G}$, by the falsity assumption above. Since $\eta\subseteq\{a_{1},\ldots,a_{i}\}\cup\tau$, that means $\emptyset\neq\eta\in\Delta_{G}$ and some action of $\eta$ must move off $\mathop{\rm src}(\eta)$ in $G$, a contradiction.

• II:

Exactly one action in $\eta^{\prime}$ has source $\diamond$:

Suppose the source of the corresponding action in $G$ is $w$. Then $w\in W$. Let $\kappa=\eta\cup\gamma$, with $\gamma\in\Delta_{H}$ a strategy that attains $w$ from anywhere in $W\mskip-2.0mu$ using actions of $H$. Then $\mathop{\rm src}(\kappa)=\left(\mathop{\rm src}(\eta^{\prime})\setminus\{\diamond\}\right)\cup W$. Since no action of $\eta^{\prime}$ moves off $\mathop{\rm src}(\eta^{\prime})$ in $\mskip 0.5muG/W\mskip-1.5mu$ and since $\gamma$ has all its sources and targets in $W$, no action of $\kappa$ moves off $\mathop{\rm src}(\kappa)$ in $G$.

Since $\{a^{\prime}_{1},\ldots,a^{\prime}_{i-1}\}\cup\tau^{\prime}\in\Delta_{G/W}$ and $\gamma\in\Delta_{H}$, Lemma 5 on page 5 implies that $\{a_{1},\ldots,a_{i-1}\}\cup\tau\cup\gamma\in\Delta_{G}$. By the falsity assumption, $\{a_{1},\ldots,a_{i}\}\cup\tau\cup\gamma\in\Delta_{G}$. Now $\kappa\subseteq\{a_{1},\ldots,a_{i}\}\cup\tau\cup\gamma$, so $\emptyset\neq\kappa\in\Delta_{G}$, and some action of $\kappa$ must move off $\mathop{\rm src}(\kappa)$ in $G$, again a contradiction.

∎

Combining Informative Action Release Sequences:

The next lemma shows how one may combine an iars in a graph with an iars from a subgraph. The subsequent corollary leverages this result with those discussed earlier, showing how one may combine an iars from a quotient graph with an iars from a fully controllable subgraph. That combinability forms a stepping stone in several proofs during the rest of the report.

Lemma 7** (Combining Graph and Subgraph Informative Action Release Sequences).**

Let $G=(V,{\mathfrak{A}})$ be a graph and let $H=(W,{\mathfrak{B}})$ be a subgraph of $G$ (with both $\mskip 1.0muV\mskip-4.5mu$ and $\mskip 1.2muW\mskip-4.0mu$ nonempty).

Suppose $a_{1},\ldots,a_{k}$, with $k\geq 1$, is an iars for $G$, such that:

• (i)

$a_{i}\in{\mathfrak{A}}\setminus{\mathfrak{B}}$, for $i=1,\ldots,k$, and

• (ii)

$\{a_{1},\ldots,a_{k}\}\cup\tau\in\Delta_{G}$, for every $\tau\in\Delta_{H}$.

Suppose $b_{1},\ldots,b_{\ell}$ is an iars for $H$, with $\{b_{1},\ldots,b_{\ell}\}\in\Delta_{H}$ and $\kern 1.49994pt\ell\geq 1$.

Then $\mskip 1.0mua_{1},\ldots,a_{k},\kern 0.80002pt{}b_{1},\ldots,b_{\ell}$ is an iars for $G$, with $\{a_{1},\ldots,a_{k},\kern 0.80002pt{}b_{1},\ldots,b_{\ell}\}\in\Delta_{G}$.

Comment: The lemma also holds when $k=0$, meaning every iars for $H$ is also an iars for $G$.

Proof.

Suppose the iars part of the assertion is false.  Let $A$ be $G$’s action relation.

Then, for some $i\in\{0,1,\ldots,\ell-1\}$,  $b_{i+1}\in(\phi_{A}\circ\psi_{A})(\{a_{1},\ldots,a_{k},\kern 0.80002pt{}b_{1},\ldots,b_{i}\})$.

(When $i=0$, this notation means $b_{1}\in(\phi_{A}\circ\psi_{A})(\{a_{1},\ldots,a_{k}\})$.)

Consequently, every maximal simplex of $\Delta_{G}$ containing $\{a_{1},\ldots,a_{k},\kern 0.80002pt{}b_{1},\ldots,b_{i}\}$ also contains $b_{i+1}$.

Since $b_{1},\ldots,b_{\ell}$ is an iars for $H$’s action relation, there exists a maximal simplex $\tau\in\Delta_{H}$ such that $\{b_{1},\ldots,b_{i}\}\subseteq\tau$ but $\{b_{1},\ldots,b_{i+1}\}\not\subseteq\tau$.

By assumption, $\{a_{1},\ldots,a_{k}\}\cup\tau\in\Delta_{G}$. Consequently, $\{a_{1},\ldots,a_{k},b_{i+1}\}\cup\tau\in\Delta_{G}$. Thus \tau\cup\{b_{i+1}\}={\mathfrak{B}}\cap\big{(}\{a_{1},\ldots,a_{k},b_{i+1}\}\cup\tau\big{)}\in\Delta_{H}, contradicting the maximality of $\mskip 1.0mu\tau$ in $\Delta_{H}$. ∎

Corollary 8** (Lifting and Combining Informative Action Release Sequences).**

Let $G=(V,{\mathfrak{A}})$ be a graph and let $H=(W,{\mathfrak{B}})$ be a fully controllable subgraph of $G$ with  $\emptyset\neq W\subsetneq V$.

Suppose $a^{\prime}_{1},\ldots,a^{\prime}_{k}$ is an iars for $G/W$, with $\{a^{\prime}_{1},\ldots,a^{\prime}_{k}\}\in\Delta_{G/W}$ and $\kern 1.49994pt{}k\geq 1$.

Suppose further that $\kern 0.80002pt{}b_{1},\ldots,b_{\ell}$ is an iars for $H$, with $\{b_{1},\ldots,b_{\ell}\}\in\Delta_{H}$ and $\kern 1.49994pt\ell\geq 1$.

Then $\mskip 1.0mua_{1},\ldots,a_{k},\kern 0.80002pt{}b_{1},\ldots,b_{\ell}$ is an iars for $G$, with $\{a_{1},\ldots,a_{k},\kern 0.80002pt{}b_{1},\ldots,b_{\ell}\}\in\Delta_{G}$.

Proof.

By Lemma 6, $a_{1},\ldots,a_{k}$ is an iars for $G$. By Lemma 5, $\{a_{1},\ldots,a_{k}\}\cup\tau\in\Delta_{G}$, for every $\tau\in\Delta_{H}$. Since actions of $H$ become self-loops in $G/W$, $a_{i}\not\in{\mathfrak{B}}$, for $i=1,\ldots,k$. The desired result therefore follows from Lemma 7. ∎

Comment:  The corollary also holds if one of $k$ or $\ell$ is [math].

4 The Nondeterministic Setting

The aim of this section is to prove Theorem 4 from page 4 for the case in which the graph $G$ is pure nondeterministic. Throughout, this section assumes that all graphs are pure nondeterministic, meaning each action is either deterministic or nondeterministic (but not stochastic).  First, we need some additional definitions and results.

4.1 Hierarchical Cyclic Graphs

We start with a recursive definition:

Definition 9** (Hierarchical Cyclic Graph).**

A pure nondeterministic graph $\mskip 3.0muG=(V,{\mathfrak{A}})$ is a hierarchical cyclic graph if one of conditions (i) or (ii) holds:

1. (i)

$\lvert V\rvert=1$* and $\mskip 4.5mu{\mathfrak{A}}=\emptyset$.* 2. (ii)

There exist $\,V_{1},\ldots,V_{k}$, $\,{\mathfrak{A}}_{1},\ldots,{\mathfrak{A}}_{k}$, $\,a_{1},\ldots,a_{k}$, with $k>1$, such that:

1. (a)

$V_{1},\ldots,V_{k}$* are nonempty pairwise disjoint subsets of $\mskip 4.0muV\!$ and $\,V\!=\bigcup_{i=1}^{k}V_{i}$.* 2. (b)

${\mathfrak{A}}_{i}$* consists of all actions in $\,{\mathfrak{A}}$ whose sources and targets lie in $V_{i}$, for $i=1,\ldots,k$.* 3. (c)

$(V_{i},\,{\mathfrak{A}}_{i})$* is a hierarchical cyclic graph, for $i=1,\ldots,k$.* 4. (d)

${\mathfrak{A}}\,=\,\{a_{1},\ldots,a_{k}\}\,\cup\mskip 4.5mu\bigcup_{i=1}^{k}{\mathfrak{A}}_{i}$. 5. (e)

*For $\kern 1.19995pt{}i=1,\ldots,k$, $\,\kern 0.80002pt\mathop{\rm src}(a_{i})\in V_{i}\,$ and $\mskip 4.0mu\mathop{\rm trg}(a_{i})\subseteq V_{i+1}$ *

• (here indices wrap around, so $V_{k+1}$ again means $V_{1}$).*

The decomposition above need not be unique. We implicitly assume a specific decomposition when stating that a graph is hierarchical cyclic. We refer to it as the tree decomposition of $\,G$.

A graph of type (i) is a leaf and a graph of type (ii) is a node.

When $\,G$ is a node, we refer to the subgraphs $\,(V_{1},{\mathfrak{A}}_{1}),\ldots,(V_{k},{\mathfrak{A}}_{k})$ in $G$’s tree decomposition as the children of $G$. Each subgraph $\mskip 1.0mu(V_{i},{\mathfrak{A}}_{i})$ is itself either a leaf or a node, with parent $(V,{\mathfrak{A}})$. When $(V_{i},{\mathfrak{A}}_{i})$ is a node, we may then speak of its children, and so forth. Transitively, we may therefore speak of all the nodes and leaves within $G$  (that includes $(V,{\mathfrak{A}})$). Finally, we may speak of the root of the tree decomposition of $\mskip 1.75muG$, meaning the node or leaf $\,(V,{\mathfrak{A}})$, i.e., $G$ itself.

For a graph of type (ii), the actions $a_{1},\ldots,a_{k}$ are the (top-level) cycle actions of $G$. Similarly, if $N\!$ is any node within $G$, the cycle actions of $N$ are the top-level cycle actions of $N\!$ when $N$ is viewed as a hierarchical cyclic graph in its own right.

Comments and Observations:

• •

Given a hierarchical cyclic graph $G$ of type (ii) as above, we can form the quotient graph $G/\{V_{1},\ldots,V_{k}\}$ (see again page 2.1). This quotient graph has state space $\{\diamond_{1},\ldots,\diamond_{k}\}$, where $\diamond_{i}$ represents all of $V_{i}$ identified to a single state, for $i=1,\ldots,k$.

All actions in each ${\mathfrak{A}}_{i}$ become nonconvergent in $G/\{V_{1},\ldots,V_{k}\}$ (actions in ${\mathfrak{A}}_{i}$ become self-loops on state $\diamond_{i}$), so we may ignore them. In contrast, each action $a_{i}$ turns into a deterministic transition  $a^{\prime}_{i}$ from state $\diamond_{i}$ to state $\diamond_{i+1}$.

We may therefore view the quotient graph $G/\{V_{1},\ldots,V_{k}\}$ as the cycle graph

[TABLE]

(the first and last states in the diagram above are the same state, namely $\diamond_{1}$).

• •

More generally, suppose $G$ is a hierarchical cyclic graph and $(W,{\mathfrak{B}})$ is some node that appears within the tree decomposition of $G$. We can form the quotient graph $\mskip 1.25muG/W$. The quotienting identifies all of $\mskip 0.5muW\mskip-2.0mu$ to a single state $\diamond$. The actions ${\mathfrak{B}}$ become self-loops on state $\diamond$. Technically, $\mskip 0.5muG/W\mskip-1.5mu$ includes these self-loops, but there is no harm ignoring them, thereby allowing us to view $G/W\mskip-2.0mu$ as a hierarchical cyclic graph. If $(W,{\mathfrak{B}})$ is $G$ itself, then we may view $\mskip 0.5muG/W\mskip-1.75mu$ as the leaf $(\{\diamond\},\emptyset)$. Otherwise, the tree decomposition of $\mskip 1.0muG/W\mskip-1.5mu$ is largely unchanged from that of $G$, except that one node, along with the subtree rooted at that node, has now become a leaf, and any actions of $G$ with source or target states in $W\mskip-1.0mu$ have had those states relabeled as $\diamond$. The only actions that become nonconvergent (by creating self-loops) are those in ${\mathfrak{B}}$, which we now ignore and discard.

• •

Conversely, suppose $(\{s\},\emptyset)$ is a leaf that appears in the tree decomposition of a hierarchical cyclic graph $G=(V,{\mathfrak{A}})$. Suppose $H=(W,{\mathfrak{B}})$ is another hierarchical cyclic graph, with states and actions distinct from those of $G$.

We can replace the leaf $(\{s\},\emptyset)$ with node $H$, to form a new hierarchical cyclic graph $\overline{G}=(\overline{V},\,\overline{{\mathfrak{A}}}\cup{\mathfrak{B}})$.

Here $\,\overline{V}\,=\,(V\setminus\{s\})\,\cup\,{W}$.  In forming $\overline{{\mathfrak{A}}}$ from ${\mathfrak{A}}$, we have some choices:

Suppose $a=v\rightarrow{T}$ is an action in ${\mathfrak{A}}$. We create a corresponding action $\overline{a}\in\overline{{\mathfrak{A}}}$ as follows:

• –

If $v=s$,  we let $\overline{v}$ be *any * state in $\mskip 0.5muW\mskip-2.0mu$ and define $\overline{a}=\overline{v}\rightarrow{T}$.

• –

If $s\in T$,  we let $S$ be *any * nonempty subset of $\mskip 1.0muW\mskip-1.5mu$ and then define $\overline{a}=v\rightarrow\overline{T}$, with $\,\overline{T}\,=\,(T\setminus\{s\})\,\cup\,{S}$.

• –

In all other cases, $\,\overline{a}=a$.

• •

A special case of the previous construction is to replace a single state $s$ in a hierarchical cyclic graph with a deterministic cycle on some new set of states, while adjusting all other actions of the encompassing graph accordingly. Actions of the encompassing graph that used to start at $s$ now start at an arbitrary state of the cycle. Actions that used to have a transition to $s$ now might transition to one or more states comprising the cycle.

• •

Every hierarchical cyclic graph is fully controllable and each of its actions is convergent.

• •

Conversely, the lemma below shows that every fully controllable pure nondeterministic graph contains a hierarchical cyclic subgraph with the same state space.  (There may be more than one such subgraph.)

Lemma 10** (Hierarchical Cyclic Subgraphs).**

Let $G=(V,{\mathfrak{A}})$, with $V\mskip-4.0mu\neq\emptyset$, be a fully controllable pure nondeterministic graph. Then $G$ contains a hierarchical cyclic subgraph $H=(V,{\mathfrak{B}})$.

Proof.

By strong induction on $\lvert V\rvert$. The base case $\lvert V\rvert=1$ is clear, so suppose $\lvert V\rvert>1$.  For every state $v$ in $V\mskip-2.0mu$ one can find a nonlooping deterministic action with target $v$ (since $G$ is fully controllable and pure nondeterministic). Backchaining such actions produces a deterministic cycle ${\mathcal{C}}$ on some subspace $W\!$ of $V$ (possibly all of $V$), containing at least two states.

Consider $G/W=(V^{\prime},{\mathfrak{A}}^{\prime})$. Here $\,V^{\prime}\,=\,(V\setminus W)\,\cup\,\{\diamond\}$, with $\diamond$ representing $W$.  $G/W$ is fully controllable (by Fact 3 on page 2.1) and pure nondeterministic, with $0<\lvert V^{\prime}\rvert<\lvert V\rvert$, so the induction hypothesis applies. We therefore obtain a hierarchical cyclic subgraph $H^{\prime}=(V^{\prime},{\mathfrak{B}}^{\prime})$ of $G/W$.

We may now replace leaf $(\{\diamond\},\emptyset)$ in $H^{\prime}$ with cycle ${\mathcal{C}}$ on state space $W$. When adjusting the encompassing actions ${\mathfrak{B}}^{\prime}$, we choose sources and targets so as to undo any relabeling of states that occurred in forming $G/W$. These adjustments produce a hierarchical cyclic subgraph $H=(V,{\mathfrak{B}})$ of $G$. ∎

4.2 Core Cycle Actions, Leaf Covers, Disruptive Sets of Actions

Suppose $H=(W,{\mathfrak{B}})$ is a hierarchical cyclic graph with $\lvert W\rvert>1$. Each state $\mskip 1.5mut$ of $\hbox{\hskip 0.4pt}{}W\!$ appears as a leaf $(\{t\},\emptyset)$ in the tree decomposition of $\mskip 1.5muH$ and has some parent node $N=(U,{\mathfrak{E}})$. Some action $c_{t}\in{\mathfrak{E}}$, necessarily a cycle action of $N$, must be deterministic with target $\mskip 1.5mut$. We refer to $c_{t}$ as $t$’s core cycle action. This action is determined uniquely by $t$ and the tree decomposition of $H$. ($H$ may contain multiple deterministic actions with target $t$, but one and only one of those actions will be a cycle action in the parent node of $(\{t\},\emptyset)$.)

With that construct in mind, we now make a series of definitions and observations.

Definition 11** (Core Cycle Actions).**

Let $H=(W,{\mathfrak{B}})$ be a hierarchical cyclic graph. The set ${\mathfrak{C}_{H}}$ of  core cycle actions of $H$ is

[TABLE]

Definition 12** (Leaf Covers).**

Let $N$ be a node in a hierarchical cyclic graph $H$. We say that $N$ covers only leaves in $H$ whenever each of $N$’s children is a leaf in $H$’s tree decomposition.

Definition 13** (Disruptive Sets of Actions).**

Let $H=(W,{\mathfrak{B}})$ be a hierarchical cyclic graph and suppose ${\mathfrak{D}}\subseteq{\mathfrak{B}}$. We say that ${\mathfrak{D}}$ is disruptive (in $H$) whenever the following condition is satisfied:

For every node that covers only leaves in $H$,

at least two of the node’s cycle actions are missing from ${\mathfrak{D}}$.

Observations:

• •

When a hierarchical cyclic graph $H=(W,{\mathfrak{B}})$ contains at least two states, $\lvert{\mathfrak{C}_{H}}\rvert=\lvert W\rvert$.

• •

A node covers only leaves in $H$ if and only if all the node’s cycle actions lie in ${\mathfrak{C}_{H}}$.

• •

The empty set of actions is always disruptive, even when $H$ is a leaf.

4.3 Cycle-Breaking Strategies

Sets of actions that do not contain any node’s full set of cycle actions are convergent and may be arranged informatively, as the following definition and lemmas make precise.

Definition 14** (Cycle-Breaking).**

Suppose $H=(W,{\mathfrak{B}})$ is a hierarchical cyclic graph. A set of actions $\tau\subseteq{\mathfrak{B}}$ is  cycle-breaking (in $H$)  if, for each node $N$ in the tree decomposition of $H$, $\hbox{\hskip 0.2pt}\tau\mskip-1.0mu$ does not contain all of $N$’s cycle actions.

Lemma 15** (Cycle-Breaking is Convergent).**

Suppose $\tau$ is a cycle-breaking set of actions in a hierarchical cyclic graph $H$.  Then $\tau\in\Delta_{H}$.

Proof.

By structural induction on the tree decomposition of $H$. The lemma holds if $\mskip 0.5muH\mskip-0.5mu$ is a leaf, since only $\tau=\emptyset\,$ is possible.  Otherwise, suppose the children of $\mskip 1.0muH\mskip-1.0mu$ are $\,(W_{1},{\mathfrak{B}}_{1}),\ldots,(W_{k},{\mathfrak{B}}_{k})$. Inductively, the lemma holds for the set of actions $\tau\cap{\mathfrak{B}}_{i}$ in the hierarchical cyclic graph $(W_{i},{\mathfrak{B}}_{i})$, for $i=1,\ldots,k$.  Let $\sigma$ consist of the top-level cycle actions of $\mskip 1.2muH\mskip-0.2mu$ that are in $\tau$.  Since ${H/\{W_{1},\ldots,W_{k}\}}$ is a directed cycle graph (see top of page • ‣ 4.1) and since $\tau$ is cycle-breaking, $\sigma^{\prime}\in\Delta_{H/\{W_{1},\ldots,W_{k}\}}$.  Thus, by repeated application of Lemma 5 on page 5, $\tau\in\Delta_{H}$. ∎

Caution:

Not all strategies in a hierarchical cyclic graph need be cycle-breaking (see page 25).

Lemma 16** (Cycle-Breaking is Informative).**

Suppose $\tau$ is a nonempty cycle-breaking set of actions in a hierarchical cyclic graph $H$.  Then some ordering of all the actions in $\mskip 1.0mu\tau$ is an informative action release sequence for $H$.

Proof.

The proof will associate to each leaf and node of $H$ an informative action release sequence, with the sequence associated to the root of $H$ comprising all of $\tau$.

For the purposes of this proof, it will be convenient to consider the empty sequence of actions as an informative action release sequence. Since $\tau$ is nonempty, the final sequence produced below will be nonempty, satisfying the standard requirement of page 2.2 that informative attribute release sequences be nonempty.

Base Case:  Associate to each leaf of $H$ the empty sequence.

Inductive Step:  Consider a node $N$ of $H$ and assume each child $C$ of $N$ has an associated informative action release sequence consisting of all the actions of $\tau$ that appear in the graph $C$. View $N$ as a hierarchical cyclic subgraph in its own right, and form the quotient graph $N^{\prime}$ obtained by identifying each child to a singleton state. The cycle actions of $N$ create a deterministic directed cycle in $N^{\prime}$. This cycle forms a minimal nonface in $\Delta_{N^{\prime}}$. By Lemma 1 on page 1, any sequential ordering of the directed edges comprising this cycle forms an iars for $N^{\prime}$, any proper subset of which is convergent. Let $\{a_{1},\ldots,a_{\ell}\}$ be the set of $N$’s cycle actions in $\tau$, this being $\emptyset$ with $\ell=0$ when none of $N$’s cycle actions lie in $\tau$. Since $\tau$ is cycle-breaking, the reasoning just given implies $\{a^{\prime}_{1},\ldots,a^{\prime}_{\ell}\}\in\Delta_{N^{\prime}}$ and $a^{\prime}_{1},\ldots,a^{\prime}_{\ell}$ is an iars for $N^{\prime}$.

Since the children of $N$ are fully controllable subgraphs of $N$, repeated application of Corollary 8 on page 8 shows that $a_{1},\ldots,a_{\ell},\hbox{\hskip 0.5pt}{}c_{1},\ldots,c_{m}$ is an iars for $N$, with $c_{1},\ldots,c_{m}$ being some concatenation of all the informative action release sequences associated to $N$’s children. Associate $a_{1},\ldots,a_{\ell},\hbox{\hskip 0.5pt}{}c_{1},\ldots,c_{m}$ to $N$. Observe that this iars consists of all actions of $\tau$ that appear in the graph $N$.  Associated to $H$ itself therefore is an iars consisting of all of $\tau$. ∎

4.4 Markings

Let $H=(W,{\mathfrak{B}})$ be a hierarchical cyclic graph. We will view each node of $H$ as being either *marked * or unmarked.  Each node is unmarked initially.  Later, we will define an algorithm that marks nodes according to some criteria.  Once marked, a node remains marked.

In order to consider marking a node $N$, we first require that each child of $N$ be either a leaf or an already marked node. The collection of marked nodes at any instant therefore defines a set ${\mathcal{M}}$ of maximal marked nodes, consisting of those nodes that are marked but have no marked parent.  After some nodes have been marked, ${\mathcal{M}}=\{(W_{1},{\mathfrak{B}}_{1}),\ldots,(W_{\ell},{\mathfrak{B}}_{\ell})\}$, for some $\ell\geq 1$, with the sets $W_{1},\ldots,W_{\ell}$ nonempty and pairwise disjoint. We may therefore form the quotient graph $H/\{W_{1},\ldots,W_{\ell}\}$, which we abbreviate as $H/{\mathcal{M}}$. We view $H/{\mathcal{M}}$ as a hierarchical cyclic graph, much as on page • ‣ 4.1, by discarding any actions that have become self-loops.

The process will be iterative, adding an additional node to the collection of marked nodes with each step. We abbreviate the notation by writing ${\mathcal{M}}^{(j)}$ to mean the maximal marked nodes at the $j^{\scriptstyle{th}}$ step, with $j\geq 1$, and by writing $H^{(j)}$ to mean $H/{\mathcal{M}}^{(j)}$. We also define ${\mathcal{M}}^{(0)}=\emptyset$ and $H^{(0)}=H$.

Observation:

Any node covering only leaves in $H^{(j)}$ corresponds to a node in $H$ that is not yet marked but that could be marked at the $(j+1)^{\scriptstyle{st}}$ step, and vice-versa.

4.5 Forward Projections

Strategies in a pure nondeterministic graph define partial orders. One may view those partial orders as forward projections of possible system states.

Definition 17** (A Strategy’s Partial Order).**

Let $G=(V,{\mathfrak{A}})$ be a pure nondeterministic graph. If $\mskip 1.0mu\sigma\in\Delta_{G}$, then $\sigma$ induces a partial order $\,\geq_{\sigma}\kern 0.80002pt$ on $V\!$ as follows:

For each $w,v\in V\!$, $\,w\geq_{\sigma}v\,$ if and only if either $w=v$ or there exist actions $a_{1},\ldots,a_{k}\in\sigma$, with $k\geq 1$, such that:

• (i)

$\mathop{\rm src}(a_{1})=w$,

• (ii)

$\mathop{\rm src}(a_{i+1})\in\mathop{\rm trg}(a_{i})$,  for $i=1,\ldots,k-1$,  and

• (iii)

$v\in\mathop{\rm trg}(a_{k})$.

Thus, $w\geq_{\sigma}\mskip-0.75muv$ if and only if $w$ is $v$ or the system might move from $w$ to $v$ while executing strategy $\sigma$  (in the diagram below, $a_{i}\in\sigma$, $v_{i}=\mathop{\rm src}(a_{i})$, and $v_{i+1}\in\mathop{\rm trg}(a_{i})$, for $i=1,\ldots,k$):

[TABLE]

The partial order $\geq_{\sigma}$ is well-defined since $\sigma$ cannot create any cycles.

Definition 18** (Forward Projection).**

Suppose $G=(V,{\mathfrak{A}})$ is a pure nondeterministic graph. Let $\sigma\in\Delta_{G}$ and $\,\emptyset\neq{W}\subseteq{V}$.  The  forward projection of $W\!$ under $\sigma$ is the set

[TABLE]

In other words, ${\mathcal{F}}_{\sigma}(W)$ consists of all states that the system might pass through or stop at, assuming the system starts at some state in $W\!$ and moves according to strategy $\sigma$. (In some papers, forward projection refers only to the states the system might stop at. Here, *forward projection * includes all states through which the system might move, including starting states.)

Lemma 19** (Disjoint Forward Projections — Core Cycle Actions).**

**

Let $H=(W,{\mathfrak{B}})$ be a hierarchical cyclic graph and suppose $\tau\in\Delta_{H}$.

Define $\kern 0.80002pt{\tau_{+}}=\,{\mathfrak{C}_{H}}\cap\tau$  and  ${\tau_{-}}=\,{\mathfrak{C}_{H}}\setminus\tau$.

For each $c\in{\tau_{-}}$, let $J_{c}={\mathcal{F}}_{\tau_{+}}(\{t\})$, with $t$ being the unique target of action $c$.  Then:

(a) The sets in the family $\mskip 1.0mu\{J_{c}\}_{c\mskip 1.0mu\in\mskip 1.0mu{\tau_{-}}}\!$ are pairwise disjoint.

(b) Suppose further that $\mskip 2.0mu\tau$ is disruptive. Let $c\in{\tau_{-}}$. Write $c=w\rightarrow t$. Then  $w\not\in J_{c}$.

In words: We split the core cycle actions ${\mathfrak{C}_{H}}$ of $H$ into two sets, those that lie in the strategy $\tau$ and those that do not. The first set is itself a strategy, so we can consider forward projections under that strategy. For each core cycle action that is not in $\tau$, we consider the forward projection of that action’s target state. The lemma asserts that the resulting forward projections are pairwise disjoint. Moreover, if $\mskip 2.0mu\tau$ is disruptive, then each such forward projection does not loop back far enough to include the source state of its generating core cycle action.

These properties will help us later to construct minimal nonfaces from which we can then extract an informative action release sequence that is sufficiently long to establish Theorem 4.

Proof.

(a) Let $\geq$ be the partial order induced by ${\tau_{+}}$ on $W$. Suppose $v\in J_{c}\cap J_{d}$, with $c,d\in{\tau_{-}}$. Write $c=w\rightarrow t$ and $d=u\rightarrow s$. Then $t\geq v$ and $s\geq v$. Since ${\tau_{+}}\subseteq{\mathfrak{C}_{H}}$, backchaining from $v$ produces a unique backwards path of action edges in ${\tau_{+}}$, with each edge actually being a deterministic action. (The path could be degenerate, consisting of no edges, just the state $v$.) That backwards path eventually encounters both $t$ and $s$, establishing that $t$ and $s$ are comparable. For example,  $s\rightarrow\cdots\rightarrow t\rightarrow\cdots\rightarrow v$  would establish $s\geq t$. Since core cycle actions $w\rightarrow t$ and $u\rightarrow s$ are missing from ${\tau_{+}}$, this is only possible if $s=t$, meaning $c=d$.

(b) Suppose $w\in J_{c}$, with $c\in{\tau_{-}}$ and $c=w\rightarrow t$. Arguing as in (a), we now obtain a cycle:

[TABLE]

All but one of the actions comprising this cycle lie in ${\tau_{+}}$.

The exception is $w\rightarrow t$, which lies in ${\tau_{-}}$.

All the actions comprising the cycle lie in ${\mathfrak{C}_{H}}$. Consider any action $u\rightarrow s$ of ${\mathfrak{C}_{H}}$. The depth******Here, the depth of a node or leaf in a tree is defined recursively as follows:

       The depth of the tree’s root is 0.  The depth of a child is one more than the depth of its parent. of the leaf $(\{u\},\emptyset)$ in the tree decomposition of $H$ must be greater than or equal to the depth of the leaf $(\{s\},\emptyset)$. Consequently, all the states in the cycle appear in $H$ as leaves at the same depth and with the same parent node. The cycle must therefore consist of that parent node’s cycle actions and the parent node cannot contain any other children. So, the parent node covers only leaves. Since $\tau$ is disruptive, at least two of the node’s cycle actions lie in ${\tau_{-}}$, not just one, establishing a contradiction. ∎

4.6 Quotienting until Disruption

The proof path now is to iteratively mark and quotient by nodes that prevent a strategy from being disruptive. Concurrently, one assembles several sets of actions that satisfy a property similar to the disjointness of forward projections described in Lemma 19.

Construction 20** (Acyclic Dissection).**

Let $H=(W,{\mathfrak{B}})$ be a hierarchical cyclic graph. Suppose $\tau\subseteq{\mathfrak{B}}$. An acyclic dissection $({\tau_{\circ}},\kern 0.80002pt{\tau_{+}},\kern 0.80002pt{\tau_{-}},\kern 0.80002pt\xi)\kern 0.80002pt$ of $\kern 0.80002pt\tau\kern 0.80002pt$ in $H$ is defined iteratively as follows:

Initialize $\tau^{(0)}=\tau$  and $\,\kappa^{(0)}=\emptyset$. Assume all nodes in the tree decomposition of $H$ are unmarked and initialize $H^{(0)}=H$, as per Section 4.4.

Set Done to true if $\mskip 1.0mu\tau$ is disruptive in $H$ and to false otherwise. 2. 2.

While not Done, run the following loop, starting from $j=0$:

1. (a)

At this stage, $\tau^{(j)}$ consists of actions in $H^{(j)}$ and is not disruptive in $H^{(j)}$.  Let $N$ be some unmarked node in $H$ such that the corresponding quotient node $N^{\prime}$ in $H^{(j)}$ covers only leaves and at most one of the cycle actions in $N^{\prime}$ is absent from $\tau^{(j)}$. 2. (b)

Suppose $N^{\prime}$ has $k$ cycle actions $\{c^{\prime}_{1},\ldots,c^{\prime}_{k}\}$. Discard one of these, so that the rest all lie in $\tau^{(j)}$. Without loss of generality, assume one may discard $c^{\prime}_{k}$. Now let

[TABLE]

Inductively: $\,\kappa^{(j+1)}$ consists of (unquotiented) actions in $H$.  In fact, $\kappa^{(j+1)}\subseteq\tau$. 3. (c)

Mark node $N$, then let $H^{(j+1)}$ be the quotient graph formed from the resulting maximal marked nodes, as per page 4.4, again viewed as a hierarchical cyclic graph. 4. (d)

Suppose $H^{(j+1)}=(W^{\prime},\,{\mathfrak{D}}^{\prime})$.  Let $\tau^{(j+1)}={\left\{a^{\prime}\in{\mathfrak{D}}^{\prime}\;\left|\;a\in\tau\right.\right\}}$. So $\tau^{(j+1)}$ is nearly the same as $\tau^{\prime}$, except that $\tau^{(j+1)}$ ignores any action of $\mskip 1.75mu\tau\mskip-0.1mu$ whose source and targets all lie within any one maximal marked node of $H$. (Prime notation indicates the correspondence between an action in $\mskip 1.0muH$ and its relabeled form in a quotient graph.) 5. (e)

If $\tau^{(j+1)}$ is disruptive in $H^{(j+1)}$, set Done to true. The loop ends. Otherwise, the loop continues, with $j+1$ in place of $j$. 3. 3.

If $\tau$ was already disruptive in $H$, let $H^{*}=H$, $\tau^{*}=\tau$, and ${\tau_{\circ}}=\emptyset$. Otherwise, let $H^{*}=H^{(j+1)}$, $\,\tau^{*}=\tau^{(j+1)}$, and $\,{\tau_{\circ}}=\kappa^{(j+1)}$, with $j+1$ as above when the loop ends. In either case, $\tau^{*}$ is disruptive in $H^{*}$.  Finally, let ${\mathfrak{C}}={\left\{c\in{\mathfrak{B}}\;\left|\;c^{\prime}\in{\mathfrak{C}_{H^{*}}}\right.\right\}}$. In other words, ${\mathfrak{C}}$ is the set of actions in $H$ that become core cycle actions in the quotient graph $H^{*}$. 4. 4.

Define $\xi$ as follows ($H$ contains a marked node if and only if the loop of step 2 was run):

Start with $\xi=\emptyset$. Then, for each *unmarked * node $N$ in $H$, let ${\mathcal{C}_{N}}$ be $N$’s cycle actions. If ${\mathcal{C}_{N}}\cap\tau$ is a proper subset of ${\mathcal{C}_{N}}$, add all of ${\mathcal{C}_{N}}\cap\tau$ to $\xi$. Otherwise, select an action $c$ in ${\mathcal{C}_{N}}\setminus{\mathfrak{C}}$. Add the actions ${\mathcal{C}_{N}}\setminus\{c\}$ to $\xi$.  (Why does $c$ exist? If not, let $N^{\prime}$ be the node in $H^{*}$ corresponding to $N$. It is well-defined since $N$ is unmarked. Then $N^{\prime}$ would cover only leaves in $H^{*}$ and thus $\tau^{*}$ would not be disruptive in $H^{*}$, a contradiction.) 5. 5.

Step 3 defined ${\tau_{\circ}}$. Now define $\,{\tau_{+}}=\,{\mathfrak{C}}\cap\xi$,  and $\,{\tau_{-}}=\,{\mathfrak{C}}\setminus\xi$.

Lemma 21**.**

Construction 20 produces an acyclic dissection $({\tau_{\circ}},{\tau_{+}},{\tau_{-}},\kern 0.80002pt\xi)$ of $\,\tau$ such that:

• (i)

${\tau_{+}}\subseteq\xi\kern 0.80002pt$* and $\kern 0.80002pt{\tau_{-}}\subseteq{\mathfrak{B}}$,*

• (ii)

${\tau_{\circ}}\cup\xi\subseteq\tau\kern 0.80002pt$* and $\kern 0.80002pt{\tau_{\circ}}\cap\xi=\emptyset$,*

• (iii)

${\tau_{\circ}}\cup\xi\hbox{\hskip 0.2pt}$* is cycle-breaking in $H$, and*

• (iv)

${\tau_{-}}\cap\tau=\emptyset$.

Proof.

The loop in step 2 of Construction 20 runs at most a finite number of times, since the graph $H$ is finite. As a result, an acyclic dissection $({\tau_{\circ}},{\tau_{+}},{\tau_{-}},\xi)$ of $\tau$ is well-defined by step 5.

Assertions (i), (ii), and (iii) are clear from the construction.

To establish assertion (iv), suppose $a\in{\tau_{-}}\cap\tau$. Let prime notation denote quotienting from $H$ to $\mskip 1.0muH^{*}$, with $\mskip 1.0muH^{*}$ as defined in step 3 of the construction, and assume the rest of the notation from the construction.

Then $a\in{\mathfrak{C}}$, $a\in\tau$, and $a\not\in\xi$.

So $a^{\prime}\in{\mathfrak{C}_{H^{*}}}$, implying $a\in{\mathcal{C}_{N}}$, with ${\mathcal{C}_{N}}$ the cycle actions of some unmarked node $N$ in $H$.

If ${\mathcal{C}_{N}}\cap\tau$ is proper subset of ${\mathcal{C}_{N}}$, then $a\in\xi$, by step 4 of the construction, producing a contradiction.

So ${\mathcal{C}_{N}}\cap\tau={\mathcal{C}_{N}}$. Let $c$ be the action removed in step 4 of the construction. So $c\not\in{\mathfrak{C}}$. Since $a\not\in\xi$, $a\in{\mathcal{C}_{N}}$, and ${\mathcal{C}_{N}}\!\setminus\!\{c\}\subseteq\xi$, it must be that $a=c$, but that contradicts $a\in{\mathfrak{C}}$. ∎

And here is a generalization of Lemma 19:

Lemma 22** (Disjoint Forward Projections).**

Suppose $H=(W,{\mathfrak{B}})$ is a hierarchical cyclic graph and $\tau\subseteq{\mathfrak{B}}$.  Construct $\kern 0.80002pt({\tau_{\circ}},{\tau_{+}},{\tau_{-}},\xi)$ from $\mskip 3.0mu\tau\mskip-0.5mu$ as per Construction 20.  Let $\eta={\tau_{\circ}}\cup{\tau_{+}}$.

Given $\kern 1.19995pt{}c\in{\tau_{-}}$, write $\kern 1.19995pt{}c=w\rightarrow T$ and define

[TABLE]

(The definition is sensible since $\eta$ is cycle-breaking in $H\!$ and so $\eta\in\Delta_{H}$.)

Then:

• (a)

The sets in the family $\mskip 1.0mu\{J_{c}\}_{c\mskip 1.0mu\in\mskip 1.0mu{\tau_{-}}}\!$ are pairwise disjoint.

• (b)

For each $c\in{\tau_{-}}$,  $\mathop{\rm src}(c)\not\in J_{c}$.

Proof.

Let prime notation denote quotienting from $H$ to $H^{*}$, where sensible, with $H^{*}$ as in step 3 of the construction. Write $H^{*}=(W^{\prime},{\mathfrak{D}}^{\prime})$, viewed with a tree decomposition derived from that of $H$.

Since $\xi$ arises only from cycle actions of unmarked nodes, each action in $\xi^{\prime}$ is a well-defined convergent action in ${\mathfrak{D}}^{\prime}$. By construction, $\xi^{\prime}$ is cycle-breaking in $H^{*}$, so $\xi^{\prime}\in\Delta_{H^{*}}$. Since $\xi^{\prime}\subseteq\tau^{*}$, $\xi^{\prime}$ is disruptive in $H^{*}$. Consequently, Lemma 19 applies to the graph $H^{*}$ and the disruptive strategy $\xi^{\prime}$.

In $H$, we have ${\tau_{+}}={{\mathfrak{C}}}\cap{\xi}$ and ${\tau_{-}}={\mathfrak{C}}\setminus\xi$. Therefore, each action $c\in{\tau_{+}}$ corresponds to an action $c^{\prime}\in{\tau^{{\kern 1.04996pt}\prime}_{+}}={{\mathfrak{C}_{H^{*}}}}\cap{\xi^{\prime}}$ in $H^{*}$, and each action $c\in{\tau_{-}}$ corresponds to an action $c^{\prime}\in{\tau^{{\kern 1.04996pt}\prime}_{-}}={\mathfrak{C}_{H^{*}}}\setminus\xi^{\prime}$.  (Recall the comment about “one-to-one correspondence” on page 2.1.)

Let $\geq$ be the partial order induced by $\eta$ on $W$ and let $\geq^{*}$ be the partial order induced by ${\tau^{{\kern 1.04996pt}\prime}_{+}}$ on $W^{\prime}$. Suppose $v\geq w$, with $v,w\in W$. Then $v^{\prime}\geq^{*}w^{\prime}$, with $v^{\prime},w^{\prime}\in W^{\prime}$ being the state relabelings of $v$ and $w$, respectively. (Why? If there is a path of action edges from $v$ to $w$ with the actions drawn from $\eta$, then there is a path of action edges from $v^{\prime}$ to $w^{\prime}$ with the actions drawn from $\eta^{\prime}$. Some of the action edges between states in $W\mskip-2.3mu$ may become self-loops when sources and targets are relabeled as states in $W^{\prime}$. Indeed, $v^{\prime}=w^{\prime}$ is possible even if $v\neq w$. Any such self-loops could only come from actions in ${\tau^{{\kern 0.70004pt}\prime}_{\circ}}$. Conversely, all actions in ${\tau^{{\kern 0.70004pt}\prime}_{\circ}}$ are self-loops. One discards those actions in forming $H^{*}$, leaving only ${\tau^{{\kern 1.04996pt}\prime}_{+}}$ from $\eta^{\prime}$. Thus there is a path of action edges from $v^{\prime}$ to $w^{\prime}$ with the actions drawn from ${\tau^{{\kern 1.04996pt}\prime}_{+}}$.)

It follows that $v\in J_{c}$ implies $v^{\prime}\in J_{c^{\prime}}$, with $J_{c^{\prime}}$ defined for $H^{*}$ and $\xi^{\prime}$ as in Lemma 19, now using $c^{\prime}$ in place of $c$, $\kern 0.80002pt{\tau^{{\kern 1.04996pt}\prime}_{+}}$ in place of ${\tau_{+}}$, and ${\tau^{{\kern 1.04996pt}\prime}_{-}}$ in place of ${\tau_{-}}$.  (To see this, write $c=w\rightarrow T$. The set of targets $T$ becomes a single state $t^{\prime}\in W^{\prime}$, since $c^{\prime}\in{\mathfrak{C}_{H^{*}}}$. Write $c^{\prime}=w^{\prime}\rightarrow t^{\prime}$. If $v\in J_{c}$, then $t\geq v$ for some $t\in T$, so $t^{\prime}\geq^{*}v^{\prime}$, and thus $v^{\prime}\in J_{c^{\prime}}$.)

Consequently, Lemma 19 establishes the claims of the current lemma. ∎

4.7 Alternate Development: Quotienting until Disruption

This subsection restates Construction 20 recursively without mentioning markings, then provides induction proofs of the corresponding lemmas. The key steps are the same as before. The rest of Section 4 will prove Theorem 4 for pure nondeterministic graphs using the earlier iterative construction, side-stepping any issue of strategy maximality in quotient graphs.  Section 5 will engage that issue when proving Theorem 4 for pure stochastic graphs.

Construction 23** (Alternate Construction: Acyclic Dissection).**

Let $H=(W,{\mathfrak{B}})$ be a hierarchical cyclic graph.  Suppose $\tau\subseteq{\mathfrak{B}}$. An acyclic dissection $({\tau_{\circ}},\kern 0.80002pt{\tau_{+}},\kern 0.80002pt{\tau_{-}},\kern 0.80002pt\xi)\kern 0.80002pt$ of $\kern 0.80002pt\tau\kern 0.80002pt$ in $H$ is defined recursively as follows:

• I.

Suppose $\tau$ is disruptive in $H$:

Define $\xi$ as follows, starting from $\xi=\emptyset$:

For each node $N$ in $H$, let ${\mathcal{C}_{N}}$ be $N$’s cycle actions. If ${\mathcal{C}_{N}}\cap\tau$ is a proper subset of ${\mathcal{C}_{N}}$, add all of ${\mathcal{C}_{N}}\cap\tau$ to $\xi$. Otherwise, there is at least one action $c$ in ${\mathcal{C}_{N}}\setminus{\mathfrak{C}_{H}}$. (If not, then $N$ would cover only leaves in $H$ and thus $\tau$ would not be disruptive.)  Pick one such action $c$ and add the remaining actions ${\mathcal{C}_{N}}\setminus\{c\}$ to $\xi$. 2. 2.

Let $\,{\tau_{\circ}}=\emptyset$, $\,{\tau_{+}}=\,{\mathfrak{C}_{H}}\cap\xi$,  and $\,{\tau_{-}}=\,{\mathfrak{C}_{H}}\setminus\xi$.

• II.

Suppose $\tau$ is not disruptive in $H$:

Let $N=(U,{\mathfrak{E}})$ be a node in $H$ that covers only leaves and at most one of whose cycle actions is absent from $\tau$. The actions ${\mathfrak{E}}$ are necessarily $N$’s cycle actions. Discard one of those actions, so the rest all lie in $\tau$. Denote that resulting set by ${\mathcal{C}}$. 2. 2.

Let $H^{*}=(W^{\prime},{\mathfrak{D}}^{\prime})$ be the hierarchical cyclic graph formed from the quotient graph $H/U$ by discarding self-loops. Let $\,({\tau^{{\kern 0.70004pt}\prime}_{\star}},\,{\tau^{{\kern 1.04996pt}\prime}_{+}},\,{\tau^{{\kern 1.04996pt}\prime}_{-}},\,\xi^{\prime})\,$ be a recursively constructed acyclic dissection of $\tau^{\prime}\cap{\mathfrak{D}}^{\prime}$ in $H^{*}$. (As usual, prime notation describes the correspondence between actions of $\mskip 2.5muH$ and actions of $\mskip 2.5muH/U$.) 3. 3.

Now define the sets of actions $\,{\tau_{+}}$, $\,{\tau_{-}}$, and $\,\xi$ by unquotienting, that is, by direct correspondence from the sets of actions $\,{\tau^{{\kern 1.04996pt}\prime}_{+}}$, $\,{\tau^{{\kern 1.04996pt}\prime}_{-}}$, and $\,\xi^{\prime}$, respectively.  Finally, let ${\tau_{\circ}}={\tau_{\star}}\cup{\mathcal{C}}$, with ${\tau_{\star}}$ formed from ${\tau^{{\kern 0.70004pt}\prime}_{\star}}$ by unquotienting.

Proof of Lemma 21, assuming alternate acyclic

dissection given by Construction 23:

Proof.

The construction terminates because $H$ is finite and each recursive invocation of the construction replaces a node with a leaf.

The proof of the specific assertions is by induction, with Case I of the construction defining the base case and Case II defining the inductive step:

• I:

In the base case, assertions (i), (ii), and (iii) are immediate from the construction. Assertion (iv) follows as it did in the earlier proof of Lemma 21, but now working directly with $H$ rather than needing to form a quotient.

• II:

Inductively, we assume assertions (i)–(iv) hold for an acyclic dissection $\,({\tau^{{\kern 0.70004pt}\prime}_{\star}},\,{\tau^{{\kern 1.04996pt}\prime}_{+}},\,{\tau^{{\kern 1.04996pt}\prime}_{-}},\,\xi^{\prime})\,$ of $\mskip 1.0mu\tau^{\prime}\cap{\mathfrak{D}}^{\prime}\mskip 1.0mu$ in $H^{*}$, using the notation from the construction. Then:

• (i)

${\tau^{{\kern 1.04996pt}\prime}_{+}}\subseteq\xi^{\prime}$, so ${\tau_{+}}\subseteq\xi$  and  ${\tau^{{\kern 1.04996pt}\prime}_{-}}\subseteq{\mathfrak{D}}^{\prime}$, so ${\tau_{-}}\subseteq{\mathfrak{D}}\subseteq{\mathfrak{B}}$.

• (ii)

${\tau^{{\kern 0.70004pt}\prime}_{\star}}\cup\xi^{\prime}\subseteq\tau^{\prime}\cap{\mathfrak{D}}^{\prime}$, so ${\tau_{\circ}}\cup\xi={\tau_{\star}}\cup{\mathcal{C}}\cup\xi\subseteq\tau$, since ${\mathcal{C}}\subseteq\tau$.

${\tau^{{\kern 0.70004pt}\prime}_{\star}}\cap\xi^{\prime}=\emptyset$, so ${\tau_{\star}}\cap\xi=\emptyset$.  Since all actions in ${\mathfrak{E}}$ become self-loops and are discarded when forming $H^{*}$ from $H$, $\,{\mathcal{C}}\cap\xi=\emptyset$, and so ${\tau_{\circ}}\cap\xi=\emptyset$.

• (iii)

${\tau^{{\kern 0.70004pt}\prime}_{\star}}\cup\xi^{\prime}\hbox{\hskip 0.2pt}$ is cycle-breaking in $H^{*}$, so ${\tau_{\circ}}\cup\xi={\tau_{\star}}\cup{\mathcal{C}}\cup\xi$ is cycle-breaking in $H$, since ${\mathcal{C}}$ consists of a proper subset of one node’s cycle actions and since ${\tau_{\star}}\cup\xi$ does not include any of that node’s actions.

• (iv)

${\tau^{{\kern 1.04996pt}\prime}_{-}}\cap\tau^{\prime}\cap{\mathfrak{D}}^{\prime}=\emptyset$, so ${\tau_{-}}\cap\tau\cap{\mathfrak{D}}=\emptyset$. Since ${\tau_{-}}\subseteq{\mathfrak{D}}$, it follows that ${\tau_{-}}\cap\tau=\emptyset$.

∎

Proof of Lemma 22, assuming alternate acyclic

dissection given by Construction 23:

Proof.

Again by induction:

• I:

The base case follows from Lemma 19, with $\xi$ in place of $\tau$, since $\xi$ is a disruptive strategy by Lemmas 21 and 15.

• II:

Inductively, the argument is much the same as in the earlier proof of this lemma. One assumes the assertions hold for the hierarchical cyclic quotient graph $H^{*}$. In moving back to $H$, one state of $H^{*}$ turns back into a cycle of states, with all but one of the cycle actions added to ${\tau_{\star}}$ to form ${\tau_{\circ}}$. The other sets of actions in the dissection do not change as one moves back from $H^{*}$ to $H$, except for relabelings of sources and targets. Consequently, execution paths of ${\tau_{\circ}}\cup{\tau_{+}}$ (within $H$) imply execution paths of ${\tau^{{\kern 0.70004pt}\prime}_{\star}}\cup{\tau^{{\kern 1.04996pt}\prime}_{+}}$ (within $H^{*}$), thereby establishing the lemma’s assertions for $H$.

∎

4.8 Acyclic Dissection Sizes

This subsection measures the size of the set ${\tau_{\circ}}\cup{\tau_{+}}\cup{\tau_{-}}$ in an acyclic dissection.

When reading the lemma below, recall that Construction 20 marks nodes in $H$.

Lemma 24** (Subgraph Sizes).**

Suppose $H=(W,{\mathfrak{B}})$ is a hierarchical cyclic graph and $\tau\subseteq{\mathfrak{B}}$.  Let  $H^{*}=(W^{\prime},{\mathfrak{D}}^{\prime})$ and $\mskip 2.0mu{\tau_{\circ}}$ be derived from $\mskip 2.0mu\tau$ as per step 3 in Construction 20 on page 20.

For each $u\in W^{\prime}$, define $\,(W_{u},{\mathfrak{B}}_{u})$ as follows:  If $\mskip 2.0muu\in W$, let $\,(W_{u},{\mathfrak{B}}_{u})$ be the leaf $\,(\{u\},\emptyset)$ of $H$. If $\kern 0.80002pt{}u\not\in W$, let $\kern 0.80002pt(W_{u},{\mathfrak{B}}_{u})$ be the maximal marked node of $\kern 0.80002pt{}H$ for which $u$ represents $W_{u}$.

Then  $\lvert{\tau_{\circ}}\cap{\mathfrak{B}}_{u}\rvert=\lvert W_{u}\rvert-1$,  for each $u\in W^{\prime}$.

Proof.

The proof is by induction on the iteration count $j\hbox{\hskip 0.2pt}$ in the loop of Construction 20, now using $H^{(j)}$ in place of $H^{*}$, $\kappa^{(j)}$ in place of ${\tau_{\circ}}$, and with the collection of marked nodes dependent on $j$. The base case, $j=0$, corresponds to all $u$ being in $W$, for which the lemma’s assertion is clear. Inductively, suppose the lemma’s assertion is true for $H^{(j)}$.

In forming $H^{(j+1)}$ from $H^{(j)}$, one marks an unmarked node $N=(V,{\mathfrak{A}})$ of $H$ whose corresponding node $N^{\prime}$ in $H^{(j)}$ covers only leaves. So $N$ is now a maximal marked node.  Let $W^{(j)}$ be the states of $H^{(j)}$, $W^{(j+1)}$ the states of $H^{(j+1)}$, and $V^{\prime}$ the states of $N^{\prime}$. Then W^{(j+1)}=\big{(}W^{(j)}\setminus V^{\prime}\big{)}\;\cup\;\{\diamond\}, with $\diamond$ representing the states $V^{\prime}$ identified to a singleton.

Let $\{c_{1},\ldots,c_{k}\}$ and $\{(V_{1},{\mathfrak{A}}_{1}),\ldots,(V_{k},{\mathfrak{A}}_{k})\}$ be the cycle actions and children of $N$ in $H$, respectively.  Without loss of generality, $\,\kappa^{(j+1)}=\kappa^{(j)}\cup\{c_{1},\ldots,c_{k-1}\}$. One has $k=\lvert V^{\prime}\rvert>1$.

Case I:  Suppose $u\in W^{(j+1)}\cap W^{(j)}$.  Then the definition of $(W_{u},{\mathfrak{B}}_{u})$ is the same via $H^{(j+1)}$ as via $H^{(j)}$. So $\lvert\kappa^{(j)}\cap{\mathfrak{B}}_{u}\rvert=\lvert W_{u}\rvert-1$. Since $(W_{u},{\mathfrak{B}}_{u})$ is either a leaf or a marked node of $H$ at the $j^{\scriptstyle{th}}$ iteration of the loop in Construction 20, ${\mathfrak{B}}_{u}$ contains none of node $N$’s cycle actions. Thus $\kappa^{(j+1)}\cap{\mathfrak{B}}_{u}={\kappa^{(j)}\cap{\mathfrak{B}}_{u}}$ and so $\lvert\kappa^{(j+1)}\cap{\mathfrak{B}}_{u}\rvert=\lvert W_{u}\rvert-1$, inductively.

Case II:  Suppose $u=\diamond$.  Then the definition of $(W_{\diamond},{\mathfrak{B}}_{\diamond})$ via $H^{(j+1)}$ is $N$, so $W_{\diamond}=V$ and ${\mathfrak{B}}_{\diamond}={\mathfrak{A}}$. The states $V^{\prime}$ of $N^{\prime}$ in $H^{(j)}$ are in one-to-one correspondence with the children $\{(V_{i},{\mathfrak{A}}_{i})\}$ of $N$. Inductively, $\lvert\kappa^{(j)}\cap{\mathfrak{A}}_{i}\rvert=\lvert V_{i}\rvert-1$. Again, $\kappa^{(j+1)}\cap{\mathfrak{A}}_{i}=\kappa^{(j)}\cap{\mathfrak{A}}_{i}$.  Moreover,

[TABLE]

By reasoning about markings, one further knows that $\kappa^{(j)}\cap\{c_{1},\ldots,c_{k}\}=\emptyset$.  Therefore

[TABLE]

and

[TABLE]

∎

Corollary 25** (Dissection Sizes).**

Suppose $H=(W,{\mathfrak{B}})$ is a hierarchical cyclic graph and $\tau\subseteq{\mathfrak{B}}$.  Let ${\tau_{\circ}}$, ${\tau_{+}}$, ${\tau_{-}}$, and $H^{*}$ be derived from $\tau$ as per Construction 20 on page 20.

Let $n=\lvert W\rvert$ and $m=\lvert{\tau_{\circ}}\cup{\tau_{+}}\cup{\tau_{-}}\rvert$.

If  $H^{*}\mskip-1.5mu$ is a leaf, then  $m=n-1$.  If  $H^{*}\mskip-1.5mu$ is a node, then  $m=n$.

Proof.

Suppose $H^{*}$ is a leaf. Then $H^{*}=(\{u\},\emptyset)$, for some $u$, and ${\tau_{+}}={\tau_{-}}=\emptyset$. Using the notation of Lemma 24, $(W_{u},{\mathfrak{B}}_{u})$ must be all of $H$.  The lemma then implies that $m=\lvert{\tau_{\circ}}\rvert=\lvert{\tau_{\circ}}\cap{\mathfrak{B}}\rvert=\lvert{\tau_{\circ}}\cap{\mathfrak{B}}_{u}\rvert=\lvert W_{u}\rvert-1=\lvert W\rvert-1=n-1$, as claimed.

Suppose $H^{*}$ is a node. Let $W^{\prime}$ be the states of $H^{*}$. For each $u\in{W^{\prime}}$, let $(W_{u},{\mathfrak{B}}_{u})$ be defined as in Lemma 24. Since ${\tau_{\circ}}$ is formed from cycle actions in marked nodes of $H$, $\,{\tau_{\circ}}=\bigcup_{u\in{W^{\prime}}}({\tau_{\circ}}\cap{\mathfrak{B}}_{u})$. We also know that $W=\bigcup_{u\in{W^{\prime}}}W_{u}$.  Thus, by the lemma,

[TABLE]

Let ${\mathfrak{C}}$ and ${\mathfrak{C}_{H^{*}}}$ be as in Construction 20. Then ${\tau_{+}}\cup{\tau_{-}}={\mathfrak{C}}$, so $\lvert{\tau_{+}}\cup{\tau_{-}}\rvert=\lvert{\mathfrak{C}}\rvert=\lvert{\mathfrak{C}_{H^{*}}}\rvert=\lvert W^{\prime}\rvert$. Consequently, $m=\lvert{\tau_{\circ}}\rvert+\lvert{\tau_{+}}\cup{\tau_{-}}\rvert=n$, as claimed. ∎

4.9 Informative Action Release Sequences for Maximal Strategies

This subsection assembles the previous results to prove Theorem 4 for pure nondeterministic graphs.

Lemma 26** (Minimal Nonfaces Overlapping Forward Projections).**

Let $G=(V,{\mathfrak{A}})$ be a fully controllable pure nondeterministic graph with $V\mskip-4.0mu\neq\emptyset$ and suppose $H=(V,{\mathfrak{B}})$ is a hierarchical cyclic subgraph of $G$.  (Recall that $H\!$ exists, by Lemma 10 on page 10.)

Let $\sigma$ be a maximal strategy in $\Delta_{G}$,  define $\tau=\sigma\cap{\mathfrak{B}}$,  let $({\tau_{\circ}},{\tau_{+}},{\tau_{-}},\xi)$ be an acyclic dissection of $\mskip 3.0mu\tau\mskip-0.25mu$ obtained from Construction 20,  and set $\,\eta={\tau_{\circ}}\cup{\tau_{+}}$.

Suppose $c\in{\tau_{-}}$.  Write $c=w\rightarrow T$ and define

[TABLE]

Then $\,{\mathcal{K}}_{c}\neq\emptyset\,$ and $\,\kappa\cap\sigma_{c}\neq\emptyset\,$ for every $\kappa\in{\mathcal{K}}_{c}$.

Comment:

The lemma tells us that every action $c\in{\tau_{-}}$ is part of a minimal nonface of $\Delta_{G}$ whose remaining actions lie in $\sigma$, and that at least one of those actions has its source, but not all its targets, in the forward projection of $c$’s targets. Here the forward projection is based on those actions of $\mskip 2.0mu\sigma$ that lie in the acyclic dissection sets $\mskip 1.0mu{\tau_{\circ}}$ and ${\tau_{+}}$. Intuitively, it is useful to think of $\mskip 0.5muH\mskip-1.5mu$ as a single node defining a directed cycle, with the projection of $\mskip 1.5mu\sigma\mskip 0.5mu$ onto that cycle being disruptive. Disruption means that the cycle splits into at least two pairwise disjoint directed arcs, as follows: The cycle edges present in $\sigma$ constitute ${\tau_{+}}$, the cycle edges missing from $\sigma$ constitute ${\tau_{-}}$, and ${\tau_{\circ}}$ is empty in this simple scenario. There is one directed arc for each action $c\in{\tau_{-}}$, starting at $c$’s target. Each arc is formed from contiguous action edges of ${\tau_{+}}$. Each arc has a forward projection flow defined on it by the directionality of those action edges. A directed arc ends when it encounters the source of another action in ${\tau_{-}}$. An arc may be degenerate, consisting of a single state. The lemma says that, for each missing cycle edge $c$, there is some action $a\in\sigma$ whose source lies in an arc that starts at $c$’s target, such that at least one of $a$’s targets lies outside this arc and such that $a$ and $c$ appear together in a minimal nonface of $\Delta_{G}$.  As we will see shortly, the “$a$ or $c$?” choice is therefore informative.

Proof.

By Lemma 21(iv) on page 21, $c\not\in\sigma$. So, since $\sigma$ is maximal in $\Delta_{G}$, $\,{\mathcal{K}}_{c}\neq\emptyset$.

Let $\kappa\in{\mathcal{K}}_{c}$ be given. Define $\gamma={\left\{a\in\kappa\;\left|\;\mathop{\rm src}(a)\in{}J_{c}\right.\right\}}$. Since $\kappa$ is a minimal nonface of $\Delta_{G}$, no action of $\kappa$ moves off $\mathop{\rm src}(\kappa)$, by Lemma 3 on page 3. Since $c\in\kappa$ and $\{c\}\in\Delta_{G}$, $\,\gamma\neq\emptyset$.  By Lemma 22(b) on page 22, $\mathop{\rm src}(c)\not\in{}J_{c}$, so $c\in\kappa\setminus\gamma$.  Thus $\emptyset\neq\gamma\subsetneq\kappa$ and $\gamma\subseteq\sigma$.

Now suppose the lemma’s second assertion is false for this $\kappa$. Then every action in $\gamma$ has all its targets in $J_{c}$. Pick some $a\in\gamma$. Since $a\in\kappa$, there exists $b\in\kappa$ such that $\mathop{\rm src}(b)\in\mathop{\rm trg}(a)\subseteq{}J_{c}$. We see therefore that $b\in\gamma$ and that no action of $\gamma$ moves off $\mathop{\rm src}(\gamma)$. Consequently, $\gamma\not\in\Delta_{G}$, which contradicts $\kappa$ being a minimal nonface of $\Delta_{G}$. ∎

Imagine revealing actions of some secret maximal strategy $\mskip 0.25mu\sigma\mskip-3.0mu\in\mskip-3.0mu\Delta_{G}\mskip 0.5mu$ to an observer who knows $G\mskip 0.85mu$ but initially merely that $\sigma$ is maximal in $\Delta_{G}$. Suppose $c\mskip 1.0mu$ is an action in ${\tau_{-}}$, as previously defined. So $c\not\in\sigma$ and $\sigma\cup\{c\}\not\in\Delta_{G}$. Let $\sigma_{c}$ be as before. The next corollary says that so long as one has not explicitly revealed any actions of $\sigma_{c}$, the observer cannot exclude the possibility that one is revealing actions of some maximal strategy other than $\sigma$, some strategy that does include action $c$. Moreover, there exists some unrevealed and unimplied action in $\sigma_{c}$ that one may yet release informatively. (The explicitly revealed actions may imply some actions in $\sigma_{c}$, but so long as none of the explicitly revealed actions themselves lie in $\sigma_{c}$, these assertions hold.)

Corollary 27** (Informative Actions in Forward Projections).**

Let the hypotheses and notation be as in Lemma 26.  In particular, $\sigma$ is maximal in $\Delta_{G}$ and $c\in{\tau_{-}}$.

Suppose $\gamma\subseteq\sigma$ such that $\gamma\cap\sigma_{c}=\emptyset$.

Let $A$ be $G$’s action relation and define $\overline{\gamma}=(\phi_{A}\circ\psi_{A})(\gamma)$.  Then:

• (i)

$\overline{\gamma}\cup\{c\}\in\Delta_{G}$.

• (ii)

${\mathcal{K}}_{c}\neq\emptyset\,$* and \,\big{(}\kappa\cap\sigma_{c}\big{)}\setminus\overline{\gamma}\;\neq\;\emptyset\, for every $\kappa\in{\mathcal{K}}_{c}$.*

Proof.

We may prove (i) by establishing that \psi_{A}\big{(}\overline{\gamma}\cup\{c\}\big{)}\neq\emptyset. By reasoning similar to that on page 122 in [6], \psi_{A}\big{(}\overline{\gamma}\cup\{c\}\big{)}=\psi_{A}\big{(}\gamma\cup\{c\}\big{)}, so it is enough to show that $\gamma\cup\{c\}\in\Delta_{G}$. Suppose this is false. Then there exists a minimal nonface $\kappa$ of $\Delta_{G}$ such that $\kappa\subseteq\gamma\cup\{c\}\subseteq\sigma\cup\{c\}$, so $\kappa\in{\mathcal{K}}_{c}$. By Lemma 26, $\kappa\cap\sigma_{c}\neq\emptyset$. That establishes a contradiction to $\gamma\cap\sigma_{c}=\emptyset$ and $c\not\in\sigma_{c}$.

Turning to (ii), ${\mathcal{K}}_{c}\neq\emptyset$ by Lemma 26.  Suppose now that $\gamma=\sigma\setminus\sigma_{c}$. Establishing the second part of (ii) for this particular $\gamma$ will establish it for all hypothesized $\gamma$, by monotonicity of closure operators. By (i), $\overline{\gamma}\cup\{c\}\in\Delta_{G}$. Since $\sigma$ is maximal in $\Delta_{G}$, $\,\overline{\gamma}\subseteq\sigma$.  Since $c\not\in\sigma$ and by Lemma 2 on page 2, $\,\kappa\setminus(\overline{\gamma}\cup\{c\})\neq\emptyset\mskip 1.5mu$ for every $\kappa\in{\mathcal{K}}_{c}$.  Since $\sigma\setminus\sigma_{c}\,=\,\gamma\,\subseteq\,\overline{\gamma}\,\subseteq\,\sigma\,$ and $\,\kappa\,\subseteq\,\sigma\cup\{c\}$, \;\kappa\setminus(\overline{\gamma}\cup\{c\})=\big{(}\kappa\cap\sigma_{c}\big{)}\setminus\overline{\gamma},  completing the proof. ∎

The following theorem has as corollary Theorem 4 of page 4 for pure nondeterministic graphs:

Theorem 28** **(Informative Action Release Sequences :

Pure Nondeterministic Graphs).

**

Let $G=(V,{\mathfrak{A}})$ be a fully controllable pure nondeterministic graph with $n=\lvert V\rvert>1$ and suppose $H=(V,{\mathfrak{B}})$ is a hierarchical cyclic subgraph of $\kern 1.19995pt{}G$.

Suppose $\sigma$ is a maximal strategy in $\Delta_{G}$. Set $\tau=\sigma\cap{\mathfrak{B}}$, then define $H^{*}$ by step 3 of Construction 20 on page 20.

• I.

If $\kern 1.19995pt{}H^{*}\mskip-0.5mu$ is a leaf, then $\kern 1.49994pt\sigma$ contains an informative action release sequence for $\kern 1.49994pt{}G$ of length at least $\kern 0.80002pt{}n-1$.

• II.

If $\kern 1.19995pt{}H^{*}\mskip-0.5mu$ is a node, then $\kern 1.49994pt\sigma$ contains an informative action release sequence for $\kern 1.49994pt{}G$ of length at least $n$.

Proof.

Throughout the proof we assume notation as given in Construction 20 and Lemma 26. Observe that $\sigma\mskip-1.0mu\neq\emptyset$, since $G$ is fully controllable with $n>1$.

I. Suppose $H^{*}$ is a leaf. Then ${\tau_{+}}={\tau_{-}}=\emptyset$.  By Lemma 21 on page 21, ${\tau_{\circ}}\subseteq\sigma$ and ${\tau_{\circ}}$ is cycle-breaking in $H$; by Corollary 25 on page 25, $\lvert{\tau_{\circ}}\rvert=n-1$; and by Lemma 16 on page 16, one may find an ordering of the actions in ${\tau_{\circ}}$ such that they form an informative action release sequence for $H$. This sequence is also informative for $G$ by the comment after Lemma 7 on page 7.

II. Suppose $H^{*}$ is a node. As in part I, one may find an ordering of the actions in ${\tau_{\circ}}\cup{\tau_{+}}$ such that they form an informative action release sequence for $G$ (${\tau_{\circ}}\cup{\tau_{+}}\neq\emptyset$, by maximality of $\sigma$ and full controllability of $G$). Write this sequence as $a_{1},\ldots,a_{\ell}$. It is contained in $\sigma$.

By Corollary 25, $\lvert{\tau_{\circ}}\cup{\tau_{+}}\cup{\tau_{-}}\rvert=n$. Since $\tau^{*}$ is disruptive in $H^{*}$, ${\tau_{-}}\neq\emptyset$. Of course, one cannot release the actions in ${\tau_{-}}$, since they are not in $\sigma$. Instead, as we will see shortly, for each $c\in{\tau_{-}}$ one may release some action of $\sigma_{c}$ informatively, thereby completing the proof.

First, observe that \big{(}{\tau_{\circ}}\cup{\tau_{+}}\big{)}\cap\sigma_{c}=\emptyset, for every $c\in{\tau_{-}}$. To see this, write $\eta={\tau_{\circ}}\cup{\tau_{+}}$ and suppose $a\in\eta$ and $\mathop{\rm src}(a)\in J_{c}$ for some $c\in{\tau_{-}}$. Write $c=w\rightarrow T$. Let $\geq$ be the partial order induced on $\mskip 1.0mu{V}$ by $\eta$. Then $t\geq\mathop{\rm src}(a)$ for some $t\in T$. Since $a\in\eta$, $\mathop{\rm src}(a)\geq s$ for every $s\in\mathop{\rm trg}(a)$. So $t\geq s$ for every $s\in\mathop{\rm trg}(a)$, meaning $\mathop{\rm trg}(a)\subseteq J_{c}$. Consequently, $a\not\in\sigma_{c}$.

Inductively, suppose we have released, for some sequence of distinct actions $c_{1},...,c_{k}$ in ${\tau_{-}}$, with $k\geq 0$, a corresponding sequence of distinct actions $b_{1},...,b_{k}$ in $\sigma$, such that $b_{i}\in\sigma_{\!c_{\scriptstyle{i}}}$, for $i=1,\ldots,k$, and such that the overall sequence $a_{1},\ldots,a_{\ell},\kern 0.80002pt{}b_{1},\ldots,b_{k}$ is an iars for $G$. If $k=\lvert{\tau_{-}}\rvert$, we are done. Otherwise, we need to show how to extend this sequence.

Let $\gamma=\{a_{1},\ldots,a_{\ell},\kern 0.80002pt{}b_{1},\ldots,b_{k}\}$ and $\overline{\gamma}=(\phi_{A}\circ\psi_{A})(\gamma)$, with $A$ being $G$’s action relation. Pick some $c\in{\tau_{-}}\setminus\{c_{1},\ldots,c_{k}\}$. We already observed that \big{(}{\tau_{\circ}}\cup{\tau_{+}}\big{)}\cap\sigma_{c}=\emptyset. By construction, $\mathop{\rm src}(b_{i})\in J_{\mskip-0.5muc_{\scriptstyle{i}}}$, for $i=1,\ldots,k$. By part (a) of Lemma 22 on page 22, $J_{\mskip-0.5muc_{\scriptstyle{i}}}\cap J_{c}=\emptyset$, for $i=1,\ldots,k$. Consequently, $\gamma\cap\sigma_{c}=\emptyset$. By Corollary 27, there exist $\kappa\in{\mathcal{K}}_{c}$ and b\in\big{(}\kappa\cap\sigma_{c}\big{)}\setminus\overline{\gamma}, so $b$ may be released informatively.  Let  $c_{k+1}=c\,$ and $\,b_{k+1}=b$. ∎

4.10 Examples for Pure Nondeterministic Graphs

This subsection shows how the proof of Theorem 28 produces informative action release sequences for various pure nondeterministic graphs and strategies.

4.10.1 A Hierarchical Pure Nondeterministic Graph

The first example considers the pure nondeterministic graph of Figure 15 on page 15. The graph may be viewed directly as a hierarchical cyclic graph, as indicated by Figure 21.

Let us consider two maximal strategies and see how our constructions generate informative action release sequences using the hierarchical cyclic graph $H$. Since $H$ and $G$ have the same actions, $\tau$ in Construction 20 on page 20 is the maximal strategy under consideration.

$\tau=\{e_{2},e_{3},b_{4}\}$

(This strategy converges to state #1.)

• •

Not used by the construction, but just for reference: $\tau$ is cycle-breaking in $H$.

• •

$\tau$ is not disruptive in $H$, so we run the loop of step 2 in Construction 20:

First we mark node $N$, defining $\kappa^{(1)}=\{e_{2},e_{3}\}$. 2. 2.

Then we mark node $H$, defining $\kappa^{(2)}=\{e_{2},e_{3},b_{4}\}$.

• •

At step 3, $H^{*}$ is a leaf. So ${\tau_{\circ}}=\kappa^{(2)}=\tau$ and ${\tau_{+}}={\tau_{-}}=\emptyset$.

• •

The proof of Lemma 16 on page 16 now produces either the sequence $\;b_{4},e_{2},e_{3}\,$ or the sequence $\,b_{4},e_{3},e_{2}\,$ as an informative action release sequence.

$\tau=\{e_{1},e_{2},a_{2}\}$

(This strategy converges to the set of states $\{3,4\}$.)

• •

$\tau$ is cycle-breaking in $H$.

• •

$\tau$ is not disruptive in $H$, so we run the loop of step 2 in the construction:

First we mark node $N$, defining $\kappa^{(1)}=\{e_{1},e_{2}\}$. 2. 2.

Then we mark node $H$, defining $\kappa^{(2)}=\{e_{1},e_{2},a_{2}\}$.

• •

At step 3, $H^{*}$ is a leaf. So ${\tau_{\circ}}$ is again all of $\tau$ and ${\tau_{+}}={\tau_{-}}=\emptyset$.

• •

Again, one may release the actions of ${\tau_{\circ}}$ informatively, as per the proof of Lemma 16, for instance as the sequence $\,a_{2},e_{1},e_{2}$.

Comments: (i) $G$’s action relation in Figure 16 on page 16 shows that no maximal strategy is disruptive in $H$, so Construction 20 will always run the loop of step 2. (ii) The construction will always assemble the entire strategy as an iars. In fact, as Figure 16 shows, the strategy complex $\Delta_{G}$ is a triangulation of $\mathbb{S}^{2}$, and in particular has no free faces. Consequently, any ordering of the actions in a maximal strategy will be an informative action release sequence for $G$.

4.10.2 A Pure Nondeterministic Graph with Several Nondeterministic Actions

Let us add some nondeterministic actions to the previous graph, as shown in Figure 22.

The earlier hierarchical cyclic graph $H$ of Figure 21 is a subgraph of the new $G$, on the same state space (but with fewer actions), so we can use the same $H$ as before to construct informative action release sequences for maximal strategies, now in the new $G$. Almost every maximal strategy in the new $\Delta_{G}$ is either identical to or a proper superset of a maximal strategy in the old $\Delta_{G}$. Intersecting one of these strategies with the actions of $H$, as Theorem 28 requires, therefore produces the same constructions as before.

There is one exception: The new $\Delta_{G}$ contains a maximal strategy, namely $\sigma_{5}$, that does not restrict to a maximal strategy in the old $\Delta_{G}$.  Let us look at that strategy more carefully:

$\sigma=\sigma_{5}=\{e_{1},a_{1},a_{2},a_{3}\}$

(This strategy converges to state #4.)

• •

$\tau$ is the intersection of $\sigma$ with the actions of $H$, so $\tau=\{e_{1},a_{2}\}$.

• •

$\tau$ is cycle-breaking in $H$.

• •

Now $\,\tau$  *is * disruptive in $H$, so Construction 20  *does not * run the loop of step 2, but skips directly to step 3.

• •

At step 3, $H^{*}$ is all of $H$, so ${\tau_{\circ}}=\emptyset$.

• •

${\mathfrak{C}}$ consists of all the core cycle actions of $H$, so ${\mathfrak{C}}=\{e_{1},e_{2},e_{3},a_{2}\}$.

• •

The construction of $\xi$ in step 4 incorporates all of $\tau$, starting from $\xi=\emptyset$, as follows:

For node $N$, step 4 adds action $e_{1}$ to $\xi$. 2. 2.

For node $H$, step 4 adds action $a_{2}$ to $\xi$.

• •

Thus ${\tau_{+}}={\mathfrak{C}}\cap\xi=\{e_{1},a_{2}\}$ and ${\tau_{-}}={\mathfrak{C}}\setminus\xi=\{e_{2},e_{3}\}$.

• •

The actions of ${\tau_{+}}$ may be released informatively in depth order, as the sequence $\,a_{2},e_{1}$.

• •

For each action in ${\tau_{-}}$, one finds an action in $\sigma$ as per the proof of Theorem 28:

For action $e_{2}\in{\tau_{-}}$, action $a_{3}\in\sigma$ lies “downstream” from $e_{2}$, forms a minimal nonface with $e_{2}$, and is not implied by $\{a_{2},e_{1}\}$. 2. 2.

For action $e_{3}\in{\tau_{-}}$, action $a_{1}\in\sigma$ lies “downstream” from $e_{3}$, forms a minimal nonface with $e_{3}$, and is not implied by $\{a_{2},e_{1},a_{3}\}$.

(The term “downstream” refers to the partial order determined by $\eta={\tau_{\circ}}\cup{\tau_{+}}$.  Since ${\tau_{\circ}}=\emptyset$, that simply means ${\tau_{+}}$ here.  Specifically, the phrase “action $b\mskip 1.0mu$ lies downstream from action $a$” means that $b$’s source lies in the forward projection of $a$’s targets under $\eta$, that is, “$\mskip 2.0mut\geq_{\eta}\mathop{\rm src}(b)$, for some $\mskip 1.0mut\in\mathop{\rm trg}(a)$”.

Moreover, this and subsequent examples, following the proof of Theorem 28, further choose $\mskip 1.0mub\mskip 1.75mu$ so that *not * all of $\kern 0.80002pt{}b$’s targets lie within the forward projection of $a$’s targets.)

Consequently, one may arrange all four actions of $\sigma$ ($=\sigma_{5}$) into an informative action release sequence for $G$. This is consistent with Theorem 28, since $H^{*}$ is a node in the construction. For instance, the sequence $\,a_{2},e_{1},a_{3},a_{1}\,$ is an iars. There are other orderings that will also produce iars of length 4, but not all will do so. For instance, releasing action $a_{1}$ as the first action in a sequence would limit the length of that sequence as an iars to 2. See [6] for further discussion of this example.

4.10.3 A Directed Graph with Several Cycles, Represented Hierarchically

Consider the directed graph $G$ of Figure 23. All the actions in this graph are deterministic. The graph has several directed cycles in it, giving us the opportunity to explore more than one hierarchical decomposition for $G$. The figure also shows a maximal strategy $\sigma$ in $\Delta_{G}$. We will focus on this one strategy, using two different hierarchical cyclic subgraphs of $G$ to construct informative action release sequences for $G$ in two different ways, such that each sequence consists of actions contained in $\sigma$.  For reference, $G$’s full action relation appears in Figure 24.

A Multi-Node Hierarchical Decomposition:

The decomposition $H$ shown in Figure 25 models $G$ directly as a hierarchical cyclic graph, meaning $H$ and $G$ contain the same states and actions. In this decomposition, the smaller two cycles of $G$ define two nodes. Each of these nodes contains only leaves, comprising the state spaces $\{1,2,3\}$ and $\{4,5,6\}$, respectively. The root of the tree has these two nodes as children, connected by a two-cycle.

$\sigma=\{e_{2},e_{5},a_{2},a_{3},a_{5},a_{6}\}$

• •

Since $H=G$, also $\,\tau=\sigma$.

• •

$\tau$ is not cycle-breaking in $H$, since it contains both cycle actions of $H$’s root node.

• •

$\tau$ is not disruptive in $H$, so Construction 20 runs the loop of step 2. The construction may mark nodes $N$ and $M$ in either order. Here we start with $N$.

Mark node $N$, defining $\kappa^{(1)}=\{a_{2},a_{3}\}$. 2. 2.

Mark node $M$, defining $\kappa^{(2)}=\{a_{2},a_{3},a_{5},a_{6}\}$. 3. 3.

Mark node $H$. Since $\tau$ contains both of $H$’s cycle actions, the construction could add either action to $\kappa^{(2)}$ in defining $\kappa^{(3)}$. Here we add $e_{2}$,  so $\kappa^{(3)}=\{a_{2},a_{3},a_{5},a_{6},e_{2}\}$.

• •

At step 3, $H^{*}$ is a leaf. So ${\tau_{\circ}}=\{a_{2},a_{3},a_{5},a_{6},e_{2}\}$ and ${\tau_{+}}={\tau_{-}}=\emptyset$.

• •

One may release the actions of ${\tau_{\circ}}$ informatively in depth order, as per the proof of Lemma 16 on page 16, for instance as the sequence $e_{2},a_{2},a_{3},a_{5},a_{6}$.

• •

Observe that ${\tau_{\circ}}$ is almost all of $\sigma$, excluding only action $e_{5}$. The construction discarded that one action when forming $\kappa^{(3)}$.

A Flat Decomposition:

Figure 26 shows another hierarchical cyclic subgraph $H$ of $\mskip 2.0muG$, on the same state space but with fewer actions. In this subgraph, the Hamiltonian cycle of $G$ defines a single node, necessarily the root of $H$, with all six states as leaves. Two of $G$’s actions do not appear in $H$.

$\sigma=\{e_{2},e_{5},a_{2},a_{3},a_{5},a_{6}\}$

• •

$\tau$ is the intersection of $\sigma$ with $H$’s actions, so $\tau=\{e_{2},e_{5},a_{3},a_{6}\}$.

• •

$\tau$ is both cycle-breaking and disruptive in $H$.

• •

Since $\tau$ is disruptive, $H^{*}=H$ and ${\tau_{\circ}}=\emptyset$ in step 3 of Construction 20.

• •

${\mathfrak{C}}$ consists of all the core cycle actions of $H$, which means all the actions of $H$ since $H$ defines a Hamiltonian cycle.  So ${\mathfrak{C}}=\{a_{1},e_{2},a_{6},a_{4},e_{5},a_{3}\}$.

• •

The construction of $\xi$ incorporates all of $\tau$, since $H$ consists of a single unmarked node. Thus ${\tau_{+}}={\mathfrak{C}}\cap\xi=\tau=\{e_{2},e_{5},a_{3},a_{6}\}$ and ${\tau_{-}}={\mathfrak{C}}\setminus\xi=\{a_{1},a_{4}\}$.

• •

The actions of ${\tau_{+}}$ may be released informatively in any order, for instance as the sequence $e_{2},e_{5},a_{3},a_{6}$.

• •

For each action in ${\tau_{-}}$, one finds an action in $\sigma$ as per the proof of Theorem 28:

(Again, “downstream” refers to the partial order determined by ${\tau_{+}}$.)

For action $a_{1}\in{\tau_{-}}$, action $a_{2}\in\sigma$ lies “downstream” from $a_{1}$, participates in the minimal nonface $\{a_{1},a_{2},a_{3}\}$ with $a_{1}$, and is not implied by $\{e_{2},e_{5},a_{3},a_{6}\}$. 2. 2.

For action $a_{4}\in{\tau_{-}}$, action $a_{5}\in\sigma$ lies “downstream” from $a_{4}$, participates in the minimal nonface $\{a_{4},a_{5},a_{6}\}$ with $a_{4}$, and is not implied by $\{e_{2},e_{5},a_{3},a_{6},a_{2}\}$.

• •

Consequently, all actions of $\sigma$ may be arranged into the informative action release sequence $e_{2},e_{5},a_{3},a_{6},a_{2},a_{5}$.

Comment:

We have seen the following: (i) With $H$ as in Figure 25, Construction 20 produces an informative action release sequence for $G$ consisting of $\mskip 1.0mu5$ actions in $\sigma$. (ii) With $H$ as in Figure 26, the construction produces an informative action release sequence consisting of all $\hbox{\hskip 0.2pt}{6}$ actions in $\sigma$. These sequence lengths match the assertions of Theorem 28 on page 28.

4.10.4 A Directed Graph with a Disruptive but not Cycle-Breaking Strategy

This example will illustrate an instance in which $\tau$ contains all the cycle actions in a node during step 4 of Construction 20. Figure 27 depicts a graph $G$ and a maximal strategy $\sigma\in\Delta_{G}$. Figure 28 displays $G$’s action relation. Figure 29 shows a hierarchical cyclic subgraph $H$ of $G$, on the same state space (but with fewer actions).  (Other such subgraphs exist, of course.)

$\sigma=\{e_{2},e_{4},a_{2},b_{1},b_{2}\}$

• •

$\tau$ is the intersection of $\sigma$ with $H$’s actions, so $\tau=\{e_{2},e_{4},a_{2}\}$.

• •

$\tau$ is not cycle-breaking in $H$, since it contains both cycle actions of $H$’s root node.

• •

$\tau$ is disruptive in $H$, since it contains only one of node $N$’s three cycle actions.

• •

Since $\tau$ is disruptive, $H^{*}=H$ and ${\tau_{\circ}}=\emptyset\,$ in step 3 of Construction 20.

• •

${\mathfrak{C}}$ consists of all the core cycle actions of $H$, so ${\mathfrak{C}}=\{a_{1},a_{2},a_{3},e_{2}\}$.

• •

The construction of $\xi=\{a_{2},e_{2}\}\,$ in step 4 occurs as follows, starting from $\xi=\emptyset$:

For node $N$, ${\mathcal{C}_{N}}=\{a_{1},a_{2},a_{3}\}$ and ${\mathcal{C}_{N}}\cap\tau=\{a_{2}\}$, so one adds action $a_{2}$ to $\xi$. 2. 2.

For node $H$, ${\mathcal{C}_{H}}=\{e_{2},e_{4}\}$ and ${\mathcal{C}_{H}}\cap\tau={\mathcal{C}_{H}}$, so one must discard some action of ${\mathcal{C}_{H}}$ that is not in ${\mathfrak{C}}$. That action is $e_{4}$. One adds action $e_{2}$ to $\xi$.

• •

Consequently, ${\tau_{+}}={\mathfrak{C}}\cap\xi=\{a_{2},e_{2}\}$ and ${\tau_{-}}={\mathfrak{C}}\setminus\xi=\{a_{1},a_{3}\}$.

• •

The actions of ${\tau_{+}}$ may be released informatively in depth order, so as the sequence $\,e_{2},a_{2}$.

• •

For each action in ${\tau_{-}}$, one finds an action in $\sigma$ as per the proof of Theorem 28:

(Once again, “downstream” refers to the partial order determined by ${\tau_{+}}$.)

For action $a_{1}\in{\tau_{-}}$, action $b_{2}\in\sigma$ lies “downstream” from $a_{1}$, forms a minimal nonface with $a_{1}$, and is not implied by $\{e_{2},a_{2}\}$. 2. 2.

For action $a_{3}\in{\tau_{-}}$, action $b_{1}\in\sigma$ lies “downstream” from $a_{3}$, forms a minimal nonface with $a_{3}$, and is not implied by $\{e_{2},a_{2},b_{2}\}$.

• •

Therefore $\,e_{2},a_{2},b_{2},b_{1}\,$ is an informative action release sequence for $G$, contained in $\sigma$.

(The order matters: Revealing either $b_{1}$ or $b_{2}$ at the beginning of the sequence would narrow the set of maximal strategies consistent with the revealed action to two instantly. One then could reveal only one more action informatively before identifying $\sigma$, that action being the other “$b_{i}$” action not yet revealed. Indeed, revealing actions $\kern 0.80002pt{}b_{1}\mskip-1.75mu$ and $\,b_{2}\kern 0.80002pt$ in either order identifies the maximal strategy to be $\sigma$. Revealing action $a_{2}$ implies action $e_{2}$, while revealing $e_{2}$ at the beginning does not imply $a_{2}$. Although not part of the construction, observe that revealing $b_{1}$ would in and of itself declare the goal to be state #3.)

5 The Stochastic Setting

The aim of this section is to prove Theorem 4 from page 4 for the case in which the graph $G$ is pure stochastic. Throughout, this section assumes that all graphs are pure stochastic, meaning each action is either deterministic or stochastic (but not nondeterministic).

Caution:

Even though all actions in a pure stochastic graph are deterministic or stochastic, there may still be a component of nondeterminism in a strategy: When multiple actions have the same source state, any one of those actions might execute from that state, with the choice potentially made by an adversary. (See again the discussion of generalized control laws on page 2.1, as well as the definitions of “moves off”, “contains a circuit”, and “strategy complex”.)

5.1 Expanding Fully Controllable Subgraphs via Minimal Nonfaces

As mentioned on page 2.3, a minimal nonface of $\Delta_{G}$ in a pure stochastic graph $G$ defines an irreducible Markov chain and thus a fully controllable subgraph of $G$. If $G$ is itself fully controllable, one may construct such a minimal nonface $\kappa$ for each maximal strategy $\sigma\in\Delta_{G}$, for instance by considering some action at a goal state of $\sigma$. The actions of the minimal nonface $\kappa$ that lie within $\sigma$ then form an informative action release sequence $z$ for $G$, contained in $\sigma$.

One may expand the state space $\mathop{\rm src}(\kappa)$ covered by this minimal nonface by considering some action outside $\sigma$, in a manner to be discussed. This process yields a new minimal nonface and thus additional actions of $\sigma$ with which to enlarge the informative action release sequence $z$. Repeating this process one may eventually encounter a situation in which there are no further useful actions outside $\sigma$. Instead, one forms a quotient graph by identifying all the states covered thus far. Recursively, one obtains an informative action release sequence within this quotient graph. Patching the two sequences together gives an overall informative action release sequence contained in $\sigma$ of length one less than the number of states in $G$’s state space.

The following construction and subsequent results describe this process formally:

Construction 29** (Minimal Nonface Expansion).**

Let $G=(V,{\mathfrak{A}})$ be a fully controllable pure stochastic graph with $n=\lvert V\rvert>1$ and suppose $\mskip 1.7mu\sigma$ is a maximal strategy in $\Delta_{G}$.

Construct a collection $\{b_{1},\ldots,b_{k}\}$ of convergent actions in ${\mathfrak{A}}$, a collection $\{\kappa_{1},\ldots,\kappa_{k}\}$ of minimal nonfaces ​of $\Delta_{G}$, a collection $\{{\mathcal{A}}_{1},\ldots,{\mathcal{A}}_{k}\}$ of subsets of $\kern 0.80002pt{\mathfrak{A}}$, and a collection $\{W_{1},\ldots,W_{k}\}$ of subsets of $\kern 1.49994pt{}V$, with $k\geq 1$, as follows:

Let $g\in V\setminus\mathop{\rm src}(\sigma)$. Choose $b_{1}\in{\mathfrak{A}}$ so that $\{b_{1}\}\in\Delta_{G}$ and $\mathop{\rm src}(b_{1})=g$. Such an action exists since $G$ is fully controllable and $\lvert V\rvert>1$. 2. 2.

Since $\sigma$ is maximal, $\sigma\cup\{b_{1}\}\not\in\Delta_{G}$, so there exists a minimal nonface $\kappa_{1}$ of $\Delta_{G}$ such that $b_{1}\in\kappa_{1}\subseteq\sigma\cup\{b_{1}\}$.   (For later reference, observe also that $\lvert\kappa_{1}\rvert>1$.) 3. 3.

Let ${\mathcal{A}}_{1}=\kappa_{1}$ and $W_{1}=\mathop{\rm src}(\kappa_{1})$. 4. 4.

Set Done to false.  While not Done, run the following loop, starting from $i=1$:

1. (a)

Consider the quotient graph $G/W_{i}$ and let prime notation refer to the correspondence between actions in $G$ and $G/W_{i}$, as per the discussion on page 2.1. 2. (b)

Define $\xi_{i}={\left\{a\in\sigma\;\left|\;\mathop{\rm src}(a)\in V\setminus W_{i}\right.\right\}}$.  So $\,\xi_{i}\subseteq\sigma$, $\,\xi_{i}\in\Delta_{G}$, and $\mathop{\rm src}(\xi_{i})\cap{W_{i}}=\emptyset$. 3. (c)

By Fact 1 on page 2.1, $\xi_{i}^{\prime}\in\Delta_{G/W_{\scriptstyle{i}}}$, so extend $\xi_{i}^{\prime}$ to a maximal simplex $\tau^{\prime}_{i}\in\Delta_{G/W_{\scriptstyle{i}}}$. 4. (d)

If $\tau_{i}\subseteq\sigma$, then set $k$ to the current value of $\kern 0.80002pt{}i$ and Done to true.  The loop ends.

Otherwise:

–

Let $b_{i+1}\in\tau_{i}\setminus\sigma$.

–

As in step 2, there exists a minimal nonface $\mskip 1.0mu\kappa_{i+1}\mskip 1.0mu$ of $\,\Delta_{G}$ such that $\mskip 1.0mub_{i+1}\in\kappa_{i+1}\subseteq\sigma\cup\{b_{i+1}\}$.  (Again, $\lvert\kappa_{i+1}\rvert>1$.)

–

Let ${\mathcal{A}}_{i+1}={\mathcal{A}}_{i}\cup\kappa_{i+1}$ and $W_{i+1}=W_{i}\cup\mathop{\rm src}(\kappa_{i+1})$.

–

The loop continues, with $i+1$ in place of $i$.

Lemma 30** (Expansive Subspaces).**

Let the hypotheses and notation be as in Construction 29.

Then $\mskip 1.0muW_{i}\subsetneq W_{i+1}$, for all $\,i$ such that $\kern 1.49994pt{}W_{i}$ and $\kern 0.80002pt{}W_{i+1}\mskip-1.0mu$ are well-defined.

(Consequently, the loop of step 4 in the construction ends, that is, $k$ is well-defined finite.)

Proof.

It is enough to show that $\mathop{\rm src}(\kappa_{i+1})\not\subseteq W_{i}$. Suppose otherwise. Since $b_{i+1}\in\kappa_{i+1}$, that would mean $\mathop{\rm src}(b_{i+1})\in W_{i}$ and $\mathop{\rm trg}(b_{i+1})\subseteq W_{i}$. (The inclusion holds because $\kappa_{i+1}$ is a minimal nonface in $\Delta_{G}$, so no action of $\kappa_{i+1}$ moves off $\mathop{\rm src}(\kappa_{i+1})$, and because $G$ is pure stochastic.) Thus $b^{{\kern 0.70004pt}\prime}_{i+1}$ would become self-looping in $G/W_{i}$, contradicting $b^{{\kern 0.70004pt}\prime}_{i+1}\in\tau^{\prime}_{i}\in\Delta_{G/W_{\scriptstyle{i}}}$. ∎

Lemma 31** (Fully Controllable Expansion).**

Let the hypotheses and notation be as in Construction 29.  Then $(W_{i},\kern 0.80002pt{\mathcal{A}}_{i})$ is a fully controllable pure stochastic graph, for $i=1,\ldots,k$.

Proof.

Observe that $k\geq 1$, since $\lvert V\rvert>1$.

Base Case:  $(W_{1},\kern 0.80002pt{\mathcal{A}}_{1})=(\mathop{\rm src}(\kappa_{1}),\,\kappa_{1})$.  Since $\kappa_{1}$ is a minimal nonface in the strategy complex of a pure stochastic graph, $(\mathop{\rm src}(\kappa_{1}),\,\kappa_{1})$ is a fully controllable pure stochastic graph.

Inductive Step:  As in the base case, $(\mathop{\rm src}(\kappa_{i+1}),\,\kappa_{i+1})$ is a fully controllable pure stochastic graph. Inductively, $(W_{i},\kern 0.80002pt{\mathcal{A}}_{i})$ is a fully controllable pure stochastic graph. Showing that $W_{i}\cap\mathop{\rm src}(\kappa_{i+1})\neq\emptyset$ would therefore establish full controllability of the pure stochastic graph $(W_{i+1},{\mathcal{A}}_{i+1})$. Suppose this intersection is empty. Then $\kappa^{\prime}_{i+1}$ is a minimal nonface in $\Delta_{G/W_{\scriptstyle{i}}}$. On the other hand, $\kappa_{i+1}\!\setminus\!\{b_{i+1}\}\subseteq\kern 0.80002pt\xi_{i}$, so $\kappa^{\prime}_{i+1}\subseteq\tau^{\prime}_{i}\in\Delta_{G/W_{\scriptstyle{i}}}$, producing a contradiction. ∎

Lemma 32** (Distinct Actions).**

Let the hypotheses and notation be as in Construction 29.

Then $\lvert\{b_{1},\ldots,b_{k}\}\rvert=k$, that is, the actions $b_{1},\ldots,b_{k}$ are distinct.

Proof.

Suppose $1\leq j\leq i<k$. Then $\mathop{\rm src}(b_{j})\in W_{j}$ and $\mathop{\rm trg}(b_{j})\subseteq W_{j}$. Since $W_{j}\subseteq W_{i}$, action $b^{\prime}_{j}$ is self-looping in $G/W_{i}$ and thus $b_{j}$ cannot be a candidate for $b_{i+1}$. ∎

Lemma 33** (Expansive Sets of Actions).**

Let hypotheses and notation be as in Construction 29. Suppose $1<i\leq k$.  Let $\ell_{i}=\lvert W_{i}\setminus W_{i-1}\rvert$.  (By Lemma 30, $\mskip 2.0mu\ell_{i}>0$.)

Then there exist actions \hbox{\hskip 1.0pt}{\mathcal{E}}_{i}\subseteq\kappa_{i}\!\setminus\!\big{(}{\mathcal{A}}_{i-1}\cup\{b_{i}\}\big{)} such that $\lvert{\mathcal{E}}_{i}\rvert=\ell_{i}$ and at most one action in $\hbox{\hskip 0.75pt}{\mathcal{E}}_{i}$ has its source in  $W_{i-1}$.  (We refer to $\mskip 3.0mu{\mathcal{E}}_{i}$ as an  expansive set of actions.)

Moreover, suppose for all \,{\mathcal{E}}\subseteq\kappa_{i}\!\setminus\!\big{(}{\mathcal{A}}_{i-1}\cup\{b_{i}\}\big{)} with $\lvert{\mathcal{E}}\rvert=\ell_{i}$, $\,\mathop{\rm src}({\mathcal{E}})\cap W_{i-1}\neq\emptyset$. Then $\mathop{\rm src}(b_{i})\not\in W_{i-1}$ and one may choose ${\mathcal{E}}_{i}$ to contain an action $e$ such that $\mathop{\rm src}(e)\in W_{i-1}$ and such that the probability of reaching $\kern 0.80002pt\mathop{\rm src}(b_{i})$ from $\kern 0.80002pt\mathop{\rm src}(e)$ under actions of $\kern 1.19995pt{\mathcal{E}}_{i}$ is nonzero.

Comments:  (a) Let ${\mathcal{A}}_{0}=\emptyset\,$, $W_{0}=\{\mathop{\rm src}(b_{1})\}$, and ${\mathcal{E}}_{1}=\kappa_{1}\!\setminus\!\{b_{1}\}$. Then the lemma holds for $i=1$, with $\mathop{\rm src}({\mathcal{E}}_{1})\cap W_{0}=\emptyset$.  (b) For $i=1,\ldots,k$, $\kern 1.19995pt{\mathcal{E}}_{i}\subseteq{\mathcal{A}}_{i}\cap\sigma$, since $\kappa_{i}\subseteq{\mathcal{A}}_{i}$ and $\kappa_{i}\!\setminus\!\{b_{i}\}\subseteq\sigma$.

Proof.

Assume $1<i\leq k$.  Readily, $\;W_{i}\setminus W_{i-1}=\mathop{\rm src}(\kappa_{i})\setminus W_{i-1}\;$ and $\;W_{i-1}=\mathop{\rm src}({\mathcal{A}}_{i-1}).$  Thus \;W_{i}\setminus W_{i-1}\;=\;\mathop{\rm src}(\kappa_{i})\setminus\mathop{\rm src}({\mathcal{A}}_{i-1})\;\subseteq\;\mathop{\rm src}\big{(}\kappa_{i}\setminus{\mathcal{A}}_{i-1}\big{)}, meaning each state in $W_{i}\setminus W_{i-1}$ is the source of some action in $\kappa_{i}$ that is not also an action in ${\mathcal{A}}_{i-1}$. If in fact every state in $W_{i}\setminus W_{i-1}$ is the source of some action in $\kappa_{i}$ that is neither an action in ${\mathcal{A}}_{i-1}$ nor the action $b_{i}$, then we may construct \mskip 1.5mu{\mathcal{E}}_{i}\subseteq\kappa_{i}\!\setminus\!\big{(}{\mathcal{A}}_{i-1}\cup\{b_{i}\}\big{)}\mskip 2.0mu such that $\mskip 1.0mu\lvert{\mathcal{E}}_{i}\rvert=\ell_{i}\mskip 2.0mu$ and $\mskip 2.0mu\mathop{\rm src}({\mathcal{E}}_{i})\cap W_{i-1}=\emptyset$.

Otherwise, since all actions in a minimal nonface have distinct sources, it is only possible to find $\ell_{i}-1$ actions in \kappa_{i}\!\setminus\!\big{(}{\mathcal{A}}_{i-1}\cup\{b_{i}\}\big{)} whose sources lie outside $W_{i-1}$. Moreover, $\mathop{\rm src}(b_{i})\in W_{i}\setminus W_{i-1}$. By the proof of Lemma 31, $\mathop{\rm src}(\kappa_{i})\cap W_{i-1}\neq\emptyset$, meaning $\kappa_{i}$ contains at least one action with source in $W_{i-1}$. We now show by backchaining from $\mathop{\rm src}(b_{i})$ how to select one such action $e$ so that ${\mathcal{E}}_{i}$ may consist of action $e$ and the $\mskip 1.5mu\ell_{i}-1$ actions just mentioned.

To reduce index clutter, we fix $i$ and make the following definitions for the rest of the proof:

[TABLE]

(By assumption for this case, $\kern 0.80002pt\lvert{\mathcal{E}}^{-}\rvert=\ell_{i}-1\kern 0.80002pt$ and $\kern 0.80002pt\mathop{\rm src}(b)\in\mathop{\rm src}(\kappa)\!\setminus\!W$.)

We now define a backchaining algorithm, with a loop index $j$, for constructing sets of actions $\emptyset\neq\tau^{(0)}\subsetneq\cdots\subsetneq\tau^{(j)}\subsetneq\cdots$. Inductively, each iteration assumes that (i) $b\in\tau^{(j)}\subsetneq\kappa$, (ii) $\tau^{(j)}\subseteq{\mathcal{E}}^{-}\cup\{b\}$, and (iii) for each $s\in\mathop{\rm src}(\tau^{(j)})$, there exists a sequence of zero or more action edges leading from $s$ to $\mathop{\rm src}(b)$, with the edges coming from actions in $\tau^{(j)}\setminus\{b\}$.

We initialize the loop with $\tau^{(0)}=\{b\}$. The loop will end by defining an action $e\kern 0.80002pt$ such that we may let ${\mathcal{E}}_{i}={\mathcal{E}}^{-}\cup\{e\}$, establishing the lemma.  The loop starts from $j=0$:

1. (a)

Since $\kappa$ is a minimal nonface in $\Delta_{G}$, with $G$ pure stochastic, $(\mathop{\rm src}(\kappa),\kern 1.19995pt\kappa)$ is a fully controllable graph in its own right and $\emptyset\neq\mathop{\rm src}(\tau^{(j)})\subsetneq\mathop{\rm src}(\kappa)$, by Lemma 3 on page 3. Thus some action $a^{(j)}\in\kappa$ moves off $\kern 1.19995pt\mathop{\rm src}(\kappa)\setminus\mathop{\rm src}(\tau^{(j)})$ in this graph. 2. (b)

If we can pick $a^{(j)}$ so that $\mathop{\rm src}(a^{(j)})\in W$, then we do so and in that case we let $e=a^{(j)}$. Either way, we define $\tau^{(j+1)}=\tau^{(j)}\cup\{a^{(j)}\}$. Condition (iii) above is satisfied by $\tau^{(j+1)}$ since it is satisfied by $\tau^{(j)}$ and $\mskip 0.5mu\mathop{\rm trg}(a^{(j)})\cap\mskip 1.1mu\mathop{\rm src}(\tau^{(j)})\neq\emptyset$. 3. (c)

If step (b) defined action $e$, then the loop ends.

Otherwise, necessarily $a^{(j)}\!\in{\mathcal{E}}^{-}\setminus\tau^{(j)}$. Thus, in this case, $\tau^{(j+1)}$ also satisfies conditions (ii) and (i) above, since in particular some action of $\kappa$ has source in $W\mskip-1.0mu$ but no action of $\tau^{(j+1)}$ does.  The loop continues, with $j+1$ in place of $j$.

By finiteness, the loop must eventually end, for some $j$. The probability of reaching $\mathop{\rm src}(b)$ from $\mathop{\rm src}(e)$ under actions of $\tau^{(j+1)}\!\setminus\!\{b\}$ is nonzero by condition (iii), so the same will be true under actions of ${\mathcal{E}}_{i}={\mathcal{E}}^{-}\cup\{e\}\subseteq\kappa$. Moreover, e\in\kappa\!\setminus\!\big{(}{\mathcal{A}}\cup\{b\}\big{)} with $\mathop{\rm src}(e)\in W$, since $e\in\kappa$ and $\emptyset\neq\mathop{\rm trg}(e)\cap\mskip 1.5mu\mathop{\rm src}(\tau^{(j)})\subseteq\mathop{\rm src}(\kappa)\setminus W$, whereas $\mathop{\rm trg}(a)\subseteq W$, for all $a\in{\mathcal{A}}$, and $\mathop{\rm src}(b)\not\in W$. ∎

For the remainder of Section 5:

Assume the hypotheses and notation of Construction 29 starting on page 29. Let ${\mathcal{A}}_{0}=\emptyset\,$ and $W_{0}=\{\mathop{\rm src}(b_{1})\}$. Define $H_{i}=(W_{i},\kern 0.80002pt{\mathcal{A}}_{i})$, for $i=0,1,\ldots,k$, with $k\geq 1$. Each $H_{i}$ is a fully controllable pure stochastic graph, by Lemma 31. Also, for $i=1,\ldots,k$, $H_{i-1}$ is a subgraph of $H_{i}$, with $\emptyset\neq{}W_{i-1}\subsetneq{}W_{i}$ and ${\mathcal{A}}_{i-1}\subsetneq{\mathcal{A}}_{i}$, by Lemmas 30 and 32, and since $\lvert\kappa_{1}\rvert>1$.  Let ${\mathcal{E}}_{1}=\kappa_{1}\!\setminus\!\{b_{1}\}$. For $i=2,\ldots,k$, define ${\mathcal{E}}_{i}$ via Lemma 33, choosing ${\mathcal{E}}_{i}$ so that $\kern 0.80002pt\mathop{\rm src}({\mathcal{E}}_{i})\cap{}W_{i-1}=\emptyset\kern 0.80002pt$ whenever possible.

Corollary 34** (Expansion Independence).**

Let hypotheses and notation be as above.

Suppose $1\leq i\leq k$.  Then $\mskip 3.0mu{\mathcal{E}}_{i}\cup\tau\in\Delta_{H_{\scriptstyle i}}$,  for every $\mskip 2.0mu\tau\in\Delta_{H_{\scriptstyle{i-1}}}$.

Proof.

If $\mathop{\rm src}({\mathcal{E}}_{i})\subseteq W_{i}\setminus W_{i-1}$, then the lemma’s assertion follows from Lemma 7.3(b)(i) in [5].

Otherwise, $i>1$. Let $e\in{\mathcal{E}}_{i}$ be as per Lemma 33. There exists a sequence of action edges

[TABLE]

for some $m\geq 1$, with $a_{j}\in{\mathcal{E}}_{i}$, $v_{j}=\mathop{\rm src}(a_{j})$, and $v_{j+1}\in\mathop{\rm trg}(a_{j})$, for all $j=1,\ldots,m$. Moreover, $v_{1}\in W_{i-1}$, $v_{j}\not\in W_{i-1}$, for $j=2,\ldots,m$, and $v_{m+1}=\mathop{\rm src}(b_{i})\not\in W_{i-1}\mskip-1.0mu\cup\mskip 0.5mu\mathop{\rm src}({\mathcal{E}}_{i})$.

Suppose ${\mathcal{E}}_{i}\cup\tau\not\in\Delta_{H_{\scriptstyle i}}$, for some $\tau\in\Delta_{H_{\scriptstyle{i-1}}}$. Let $\varrho$ be a minimal nonface of $\Delta_{H_{\scriptstyle i}}$, with $\emptyset\neq\varrho\subseteq{\mathcal{E}}_{i}\cup\tau$. Some or all of the actions $\{a_{1},\ldots,a_{m}\}$ lie in $\varrho$.  Certainly $e\in\varrho$, again by Lemma 7.3(b)(i) in [5]. Since $\varrho$ is a minimal nonface, $e$ is the only action of $\varrho$ with source $\mathop{\rm src}(e)$.  Since $\tau\in\Delta_{H_{\scriptstyle{i-1}}}$, $\kern 0.80002pt\mathop{\rm src}(\varrho\setminus{\mathcal{E}}_{i})\subseteq W_{i-1}$. The actions in ${\mathcal{E}}_{i}$ all have distinct sources. Thus no action in $\varrho$ other than $a_{j}$ (if $a_{j}$ is even in $\varrho$) can have source $v_{j}$, for $j=2,\ldots,m$.

Consequently, there is a nonzero probability that the system will transition to and stop at a state outside $\mathop{\rm src}(\varrho)$ when started at $\mathop{\rm src}(e)$, while moving under actions of $\varrho$. Some action of $\varrho$ therefore moves off $\mathop{\rm src}(\varrho)$, which is a contradiction. ∎

Corollary 35** (Cardinality of Expansive Actions).**

Let hypotheses and notation be as above.

Then \,\big{\lvert}\mskip-2.25mu\bigcup_{i=1}^{k}{\mathcal{E}}_{i}\mskip 1.0mu\big{\rvert}=\lvert W_{k}\rvert-1.

Proof.

By Construction 29, Lemma 33, and subsequent comments,

[TABLE]

∎

5.2 Informative Action Release Sequences from Expansive Sets of Actions

This subsection shows how the constructions of the previous subsection produce informative action release sequences. Some notational abbreviations will be useful:

Notation and Terminology:

Rather than merely write sequences of actions, $b_{1},\ldots,b_{m}$, we may write sequences of sets of actions ${\mathcal{B}}_{1},\ldots,{\mathcal{B}}_{m}$, assuming the sets ${\mathcal{B}}_{1},\ldots,{\mathcal{B}}_{m}$ are nonempty and pairwise disjoint.

The meaning of a set ${\mathcal{B}}_{i}$ of actions is to indicate a multiplicity of sequences of actions, one for each possible permutation of the actions in the set ${\mathcal{B}}_{i}$. The sequence of sets ${\mathcal{B}}_{1},\ldots,{\mathcal{B}}_{m}$ represents all possible orderings of the actions $\cup_{i=1}^{m}{\mathcal{B}}_{i}$ consistent with the top-level ordering ${\mathcal{B}}_{1},\ldots,{\mathcal{B}}_{m}$.  Here, “consistent” means actions in ${\mathcal{B}}_{i}$ must appear before actions in ${\mathcal{B}}_{j}$ whenever $i<j$, but the ordering is otherwise unconstrained.

For example, the sequence of sets $\,\{a,b\},\{c\},\{d,e,f\}\,$ represents 12 sequences of actions:

[TABLE] 2. 2.

We say that a sequence ${\mathcal{B}}_{1},\ldots,{\mathcal{B}}_{m}$ of sets of actions is informative for $\kern 0.80002pt{}G\kern 1.49994pt$ if each of the sequences of actions it represents is an informative action release sequence for graph $G$. 3. 3.

In place of a singleton set, we may also simply write the action it contains. For instance, we could write the top-level sequence in the example above as $\,\{a,b\},\,c,\{d,e,f\}\,$.

Lemma 36** (Expanding Informative Actions).**

Suppose $G=(V,{\mathfrak{A}})$ is a fully controllable pure stochastic graph with $n=\lvert V\rvert>1$.  Let $\sigma$ be a maximal strategy in $\Delta_{G}$.

From $G$ and $\mskip 1.0mu\sigma\mskip-1.0mu$ construct $\kern 0.80002pt{}H_{1},\ldots,H_{k}$ and $\kern 0.80002pt{\mathcal{E}}_{1},\ldots,{\mathcal{E}}_{k}$, with $k\geq 1$, as per Construction 29 on page 29, Lemma 33 on page 33, and the definitions and notation of page 5.1.

Then, for each $i$, with $1\leq i\leq k$, the sequence ${\mathcal{E}}_{i},{\mathcal{E}}_{i-1},\ldots,{\mathcal{E}}_{1}$ is informative for $H_{i}$.

Proof.

By induction on $i$.  Let $i$, with $1\leq i\leq k$, be given.

The set ${\mathcal{E}}_{i}$ is a nonempty proper subset of a minimal nonface of $\Delta_{G}$ and thus of $\Delta_{H_{\scriptstyle i}}$. By Lemma 1 on page 1, every ordering of actions in ${\mathcal{E}}_{i}$ is an informative action release sequence for $H_{i}$.  Moreover, ${\mathcal{E}}_{i}\in\Delta_{H_{\scriptstyle i}}$.

If $i=1$, these observations establish the base case.

If $i>1$, then inductively ${\mathcal{E}}_{i-1},\ldots,{\mathcal{E}}_{1}$ is informative for $H_{i-1}$ and ${\mathcal{E}}_{i-1}\cup\cdots\cup{\mathcal{E}}_{1}\in\Delta_{H_{\scriptstyle{i-1}}}$.

By Corollary 34 on page 34, ${\mathcal{E}}_{i}\cup\tau\in\Delta_{H_{\scriptstyle i}}$, for every $\tau\in\Delta_{H_{\scriptstyle{i-1}}}$. By construction, no action in ${\mathcal{E}}_{i}$ is an action in the graph $H_{i-1}$. Therefore, by Lemma 7 on page 7, ${\mathcal{E}}_{i},{\mathcal{E}}_{i-1},\ldots,{\mathcal{E}}_{1}$ is informative for $H_{i}$. Moreover, ${\mathcal{E}}_{i}\cup\cdots\cup{\mathcal{E}}_{1}\in\Delta_{H_{\scriptstyle i}}$, since, for instance, ${\mathcal{E}}_{i}\cup\cdots\cup{\mathcal{E}}_{1}\subseteq\sigma$. ∎

Corollary 37** (Expanding Informative Actions in $G$).**

Let the hypotheses and notation be as for Lemma 36.  For each $i$ with $1\leq i\leq k$, the sequence ${\mathcal{E}}_{i},{\mathcal{E}}_{i-1},\ldots,{\mathcal{E}}_{1}$ is informative for $G$.

Proof.

By the previous lemma, ${\mathcal{E}}_{i},{\mathcal{E}}_{i-1},\ldots,{\mathcal{E}}_{1}$ is informative for $H_{i}$. The comment after the statement of Lemma 7 on page 7 establishes the corollary. ∎

5.3 An Informative Action Release Sequence from a Quotient

The loop in Construction 29 may end in step 4(d) (on page 4d) with $\tau_{k}=\emptyset$. This will occur if and only if $W_{k}=V$. In that case, Corollary 37 (above) and Corollary 35 (on page 35) imply that the sequence ${\mathcal{E}}_{k},\ldots,{\mathcal{E}}_{1}$ provides an informative action release sequence for $G$ of length $n-1$, with all actions of the sequence contained in $\sigma$, and with $n=\lvert V\rvert>1$.

Otherwise, the following lemma ensures that one may add a prefix of informative actions to that sequence whenever one can find an informative sequence in the quotient graph $G/W_{k}$.

Lemma 38** (Informative Actions from Quotient).**

Suppose $G=(V,{\mathfrak{A}})$ is a fully controllable pure stochastic graph with $n=\lvert V\rvert>1$.  Let $\sigma$ be a maximal strategy in $\Delta_{G}$.

From $G$ and $\sigma\mskip-1.0mu$ construct $k$, $W_{k}$, $\tau_{k}$, $H_{k}$, and $\kern 0.80002pt{\mathcal{E}}_{1},\ldots,{\mathcal{E}}_{k}$, as per Construction 29 on page 29, Lemma 33 on page 33, and the definitions and notation of page 5.1. (Recall that $k\geq 1$.)

Suppose further that $a^{\prime}_{1},\ldots,a^{\prime}_{\ell}$ is an informative action release sequence for $G/W_{k}$, with $\ell\geq 1$ and $\{a^{\prime}_{1},\ldots,a^{\prime}_{\ell}\}\subseteq\tau^{\prime}_{k}\in\Delta_{G/W_{\scriptstyle{k}}}$.

Then $\kern 0.80002pt{}a_{1},\ldots,a_{\ell},\kern 1.19995pt{\mathcal{E}}_{k},\ldots,{\mathcal{E}}_{1}$ is informative for $\kern 0.80002pt{}G$, with all actions contained in $\sigma$.

Proof.

By construction, $\{a_{1},\ldots,a_{\ell}\}\subseteq\kern 0.80002pt\tau_{k}\subseteq\sigma$ and $\cup_{i=1}^{k}{\mathcal{E}}_{i}\subseteq\sigma$.

By Lemma 31 on page 31, $H_{k}$ is a fully controllable subgraph of $G$.  Also, $\emptyset\neq W_{k}\subsetneq V$.

By Lemma 36 on page 36, $\kern 0.80002pt{\mathcal{E}}_{k},\ldots,{\mathcal{E}}_{1}$ is informative for $H_{k}$. Any informative sequence of actions formed from $\kern 0.80002pt{\mathcal{E}}_{k},\ldots,{\mathcal{E}}_{1}$ is a subset of $\sigma$, therefore convergent in both $G$ and $H_{k}$.

Corollary 8 on page 8 therefore establishes the desired result. ∎

The following theorem instantiates Theorem 4 of page 4 for pure stochastic graphs:

Theorem 39** **(Informative Action Release Sequences :

Pure Stochastic Graphs).

**

Let $G=(V,{\mathfrak{A}})$ be a fully controllable pure stochastic graph with $n=\lvert V\rvert>1$.

Suppose $\sigma$ is a maximal strategy in $\Delta_{G}$.

Then $\kern 0.80002pt\sigma$ contains an informative action release sequence for $\kern 1.49994pt{}G$ of length at least ${\kern 1.34995pt}n-1$.

Proof.

By induction on $n$.

Base Case:  $n=2$.

In this case, $\Delta_{G}$ consists of two (nonempty) maximal strategies, one for each state in $V$. (The strategy for state $v$ consists of all actions with source $v$ that are not deterministic self-loops. The strategy converges to the other state.) Any single action in one of these strategies constitutes an informative action release sequence for $G$ and is contained in the given strategy.

Inductive Step:  $n>2$.

From $G$ and $\sigma$ construct $k$, $W_{k}$, $\tau_{k}$, and $\kern 0.80002pt{\mathcal{E}}_{1},\ldots,{\mathcal{E}}_{k}$, using Construction 29 on page 29, Lemma 33 on page 33, and subsequent comments. Recall that $k\geq 1$.

As discussed on page 5.3, if $\tau_{k}=\emptyset$, then the sequence ${\mathcal{E}}_{k},\ldots,{\mathcal{E}}_{1}$ provides an informative action release sequence for $G$ of length $n-1$, consisting of actions in $\sigma$.

Otherwise, let $\ell=\lvert V\setminus W_{k}\rvert$. Then $\ell>0$. The quotient graph $G/W_{k}$ is pure stochastic and fully controllable, by Fact 3 on page 2.1. It has state space $V^{\prime}=(V\setminus W_{k})\cup\{\diamond\}$, with $\diamond$ representing all of $W_{k}$ identified to a single state.

Since the minimal nonface $\kappa_{1}$ in Construction 29 contains at least two actions, $W_{k}$ contains at least two states. Therefore $2\leq\lvert V^{\prime}\rvert<n$. Inductively, the theorem holds for graph $G/W_{k}$ and maximal strategy $\tau^{\prime}_{k}$, producing an informative action release sequence $a^{\prime}_{1},\ldots,a^{\prime}_{\ell}$ for $G/W_{k}$ with $\{a^{\prime}_{1},\ldots,a^{\prime}_{\ell}\}\subseteq\tau^{\prime}_{k}\in\Delta_{G/W_{\scriptstyle{k}}}$. By Lemma 38, $a_{1},\ldots,a_{\ell},\kern 1.19995pt{\mathcal{E}}_{k},\ldots,{\mathcal{E}}_{1}$ is informative for $\kern 0.80002pt{}G$, with all actions contained in $\sigma$. Any consequent informative action release sequence has length $\kern 0.80002pt\ell+\lvert\cup_{i=1}^{k}{\mathcal{E}}_{i}\rvert=\lvert V\setminus W_{k}\rvert+(\lvert W_{k}\rvert-1)=n-1$, by Corollary 35 on page 35. ∎

5.4 Examples for Pure Stochastic Graphs

This subsection shows how the proof of Theorem 39 produces informative action release sequences for some pure stochastic graphs and strategies. For clarity, figures discard self-loops.

5.4.1 A Directed Graph with Several Cycles

Consider again the strongly connected directed graph $G$ and maximal strategy $\sigma$ of Figure 23 on page 23. Earlier, we viewed $G$ as a pure nondeterministic graph with different hierarchical cyclic subgraphs. Now, we view $G$ as a pure stochastic graph and apply Construction 29 to obtain an informative action release sequence of length 5 for $G$, contained in $\sigma$.

For this example, it turns out that the loop of step 4 in the construction runs once, ending with $k=1$, but without having covered the entire state space of the graph. Consequently, as indicated by Theorem 39’s inductive proof, one needs to invoke the construction again, on a quotient graph. Again, the loop runs only once. In total, there are three invocations of the construction. The synopses below show how local variables in the construction are instantiated.

In the first invocation of Construction 29, the graph is $G$ as in Figure 23 and the maximal strategy is $\sigma=\{e_{2},e_{5},a_{2},a_{3},a_{5},a_{6}\}$. For $g$, one may use either state in $\sigma$’s goal set $\{1,4\}$. Using $g=1$, one finds $b_{1}=a_{1}$, yielding the minimal nonface $\kappa_{1}=\{a_{1},a_{2},a_{3}\}$. Thus $W_{1}=\{1,2,3\}$. The comments at the top of page 5.1 produce ${\mathcal{E}}_{1}=\{a_{2},a_{3}\}$.

Running the loop of step 4 in the construction, with $i=1$, one obtains $\xi_{1}=\{e_{5},a_{5},a_{6}\}$. In $G/W_{1}$, $\kern 0.80002pt\xi^{\prime}_{1}$ has a single maximal extension, namely $\tau^{\prime}_{1}=\{e^{\prime}_{2},e^{\prime}_{5},a^{\prime}_{5},a^{\prime}_{6}\}$.  Figure 30 shows both $G/W_{1}$ and $\tau^{\prime}_{1}$, with $\Diamond$ representing all of $W_{1}$ identified to a singleton.

Since $\tau_{1}\subseteq\sigma$, the loop ends with $k=1$. 2. 2.

In the second invocation of Construction 29, the graph is $G/W_{1}$ and the maximal strategy is $\tau^{\prime}_{1}=\{e^{\prime}_{2},e^{\prime}_{5},a^{\prime}_{5},a^{\prime}_{6}\}$. The strategy has goal state $4$, so let $g=4$. Therefore, in this invocation of the construction, $b^{\mskip 1.0mu\prime}_{1}=a^{\prime}_{4}$, yielding the minimal nonface $\kappa^{\prime}_{1}=\{a^{\prime}_{4},a^{\prime}_{5},a^{\prime}_{6}\}$. (We use single prime notation to indicate actions in $G/W_{1}$, including references to local variables within this invocation of Construction 29.)

We now write $U_{1}$ for $\mathop{\rm src}(\kappa^{\prime}_{1})$, in order to avoid confusion with the earlier $W_{1}$. Thus $U_{1}=\{4,5,6\}$. The comments at the top of page 5.1 produce ${\mathcal{E}}^{\prime}_{1}=\{a^{\prime}_{5},a^{\prime}_{6}\}$.

(Below, we will now also use double prime notation, specifically to indicate actions in $(G/W_{1})/U_{1}$, including references to local variables within Construction 29. We therefore write $\xi^{\prime}_{1}$ in place of $\xi_{1}$ in step 4(b) and $\xi^{\prime\prime}_{1}$ in place of $\xi^{\prime}_{1}$ in step 4(c).)

Running the loop of step 4, with $i=1$, one obtains $\xi^{\prime}_{1}=\{e^{\prime}_{2}\}$. In $(G/W_{1})/U_{1}$, $\kern 0.80002pt\xi^{\prime\prime}_{1}$ has a single maximal extension, namely itself. We refer to that extension as $\rho^{\mskip 0.5mu\prime\prime}_{1}$, in order to avoid confusion with the earlier $\tau_{1}$. Figure 31 shows both $(G/W_{1})/U_{1}$ and $\rho^{\prime\prime}_{1}$, with $\Diamond$ as before and $\mskip 1.0mu\Box$ representing all of $U_{1}$ identified to a singleton.

Since $\rho^{\prime}_{1}\subseteq\tau^{\prime}_{1}$, the loop ends with $k=1$. 3. 3.

Since the graph $(G/W_{1})/U_{1}$ has only two states, one could now simply refer to the base case in the proof of Theorem 39. However, we will invoke Construction 29 yet a third time, with graph $(G/W_{1})/U_{1}$ and maximal strategy $\rho^{\prime\prime}_{1}=\{e^{\prime\prime}_{2}\}$. This strategy has goal state $\Box$, thus yielding minimal nonface $\kappa_{1}^{\prime\prime}=\{e^{\prime\prime}_{2},e^{\prime\prime}_{5}\}$ with source set $\{\Diamond,\Box\}$. The comments at the top of page 5.1 produce the expansive set of actions ${\mathcal{E}}^{\prime\prime}_{1}=\{e^{\prime\prime}_{2}\}$. The loop ends because the source set is the entire state space, as discussed at the beginning of Section 5.3 on page 5.3.

Finally, one assembles the various expansive sets in reverse order of the recursive invocations of Construction 29. This process produces the following sequence of sets of actions in $G$:

[TABLE]

That sequence of sets represents four informative action release sequences for $G$, each consisting of actions in $\sigma$:

[TABLE]

(We know from Section 4.10.3  that $\sigma$ also contains other informative action release sequences.)

5.4.2 A Pure Stochastic Graph

Figure 32 depicts a fully controllable pure stochastic graph $G$, along with a maximal strategy that contains an action with stochastic transitions. The strategy complex $\Delta_{G}$ appears in Figure 33, along with $G$’s action relation. The maximal strategy under consideration is $\sigma^{+}_{3}$.

Unlike in a pure nondeterministic graph, cycling is permitted in a pure stochastic graph, so long as the cycling is transient. The definition of “moves off” from page • ‣ 2.1 captures this distinction. For instance, in the current example, $\{a_{1},a_{2}\}$ is a convergent (nonmaximal) strategy, with goal state #3. The set of actions $\{a_{1},a_{2}\}$ would not be convergent if action $a_{1}$ were nondeterministic, since then an adversary could force infinite cycling between states #1 and #2. However, $a_{1}$ is stochastic, so there is a nonzero probability that the system will exit such a cycle, transitioning to state #3 instead. The precise transition probabilities of action $a_{1}$ affect expected convergence times, as discussed in [5, 4], but not overall convergence.

Given $G$ and $\sigma=\sigma^{+}_{3}=\{a_{1},a_{2},d_{2}\}$ as in Figure 32, Construction 29 computes as follows:

There is one state outside $\mathop{\rm src}(\sigma)=\{1,2\}$, so $g=3$.  Action $b_{1}$ in the construction should be an action with source $g$, meaning it is action $a_{3}$ of Figure 32. 2. 2.

One may then use minimal nonface $\kappa_{1}=\{d_{2},a_{3}\}$. (Another possibility is $\{a_{1},a_{2},a_{3}\}$.) 3. 3.

So ${\mathcal{A}}_{1}=\{d_{2},a_{3}\}$ and $W_{1}=\{2,3\}$. 4. 4.

Now the loop of the construction runs:

• $i=1$:

1. (a)

​Figure 34 depicts graph $G/W_{1}$, omitting the self-looping actions at state $\Diamond$. 2. (b)

$\xi_{1}={\left\{a\in\sigma\;\left|\;\mathop{\rm src}(a)\not\in W_{1}\right.\right\}}=\{a_{1}\}$. 3. (c)

$\xi^{\prime}_{1}$ is not maximal in $\Delta_{G/W_{\scriptstyle{1}}}$. It has unique maximal extension $\tau^{\prime}_{1}=\{a^{\prime}_{1},d^{\mskip 1.0mu\prime}_{1}\}$. 4. (d)

$\tau_{1}\not\subseteq\sigma$. Since $\tau_{1}\setminus\sigma=\{d_{1}\}$, $\,b_{2}=d_{1}$. Thus $\kappa_{2}=\{d_{1},a_{2}\}$, ${\mathcal{A}}_{2}={\mathcal{A}}_{1}\cup\kappa_{2}=\{d_{2},a_{3},d_{1},a_{2}\}$, and $W_{2}=W_{1}\cup\mathop{\rm src}(\kappa_{2})=\{1,2,3\}$.

• $i=2$:

1. (a)

Graph $G/W_{2}=(\{\Box\},\emptyset)$, with $\Box$ representing all states of $G$ identified to a singleton.  So $\Delta_{G/W_{\scriptstyle{2}}}=\{\emptyset\}$, the empty simplicial complex. 2. (b)

$\xi_{2}=\emptyset$. 3. (c)

$\xi^{\prime}_{2}$ is maximal in $\Delta_{G/W_{\scriptstyle{2}}}$, so $\tau^{\prime}_{2}=\emptyset$. 4. (d)

$\tau_{2}\subseteq\sigma$, so the loop ends, with $k=2$.

Lemma 33 and subsequent comments construct expansive sets ${\mathcal{E}}_{1}$ and ${\mathcal{E}}_{2}$ as follows (variable bindings for $g$, $W_{1}$, $W_{2}$, $b_{1}$, $b_{2}$, $\kappa_{1}$, and $\kappa_{2}$ are as above, actions $d_{2}$ and $a_{2}$ are as in Figure 32):

Let $W_{0}=\{g\}=\{3\}$. 2. 1.

Since $W_{1}\setminus W_{0}=\{2\}=\mathop{\rm src}(\kappa_{1}\!\setminus\!\{b_{1}\})$, $\;{\mathcal{E}}_{1}=\kappa_{1}\!\setminus\!\{b_{1}\}=\{d_{2}\}$.  (See also page 5.1.) 3. 2.

While $W_{2}\setminus W_{1}=\{1\}\neq\{2\}=\mathop{\rm src}(\{a_{2}\})=\mathop{\rm src}(\kappa_{2}\!\setminus\!\{b_{2}\})$, $\kern 1.19995pt\mathop{\rm src}(b_{2})\in\mathop{\rm trg}(a_{2})$, so ${\mathcal{E}}_{2}=\{a_{2}\}$.

By Corollary 37, the sequence ${\mathcal{E}}_{2},{\mathcal{E}}_{1}$ is informative for $G$, with all actions contained in $\sigma^{+}_{3}$. We thus obtain the informative action release sequence $\,a_{2},d_{2}$.  Side note: This is not the only iars contained in $\sigma^{+}_{3}$. The longest such iars consists of all actions in $\sigma^{+}_{3}$, in the order $\,a_{1},d_{2},a_{2}$.

Interpretation:

Figure 35 depicts ${\mathcal{A}}_{2}$ as a graph. The graph consists of two independent two-cycles. Relationally, we may think of these two two-cycles as two independent bits of information, forming a basis for informative action release sequences. In more general examples (see Section 5.4.3), there may be less independence. Consequently, Lemma 33 (page 33) constructs expansive sets of actions, which Lemma 36 (page 36) then arranges informatively.

5.4.3 A Pure Stochastic Graph Highlighting Expansive Set Order

We now apply Construction 29 to the pure stochastic graph $G$ and maximal strategy $\sigma\in\Delta_{G}$ of Figure 36. We may let $g=1$.  Then $\mskip 1.0mub_{1}=c_{1}$, $\mskip 3.0mu\kappa_{1}=\{c_{1},a_{2},e_{3}\}$, and $W_{1}=\{1,2,3\}$. Thus ${\mathcal{E}}_{1}=\{a_{2},e_{3}\}$. Now $\xi_{1}=\emptyset$, so there is a choice in constructing $\tau^{\prime}_{1}$. If we choose $\tau^{\prime}_{1}=\{a^{\prime}_{4}\}$, then $\tau_{1}\not\subseteq\sigma$ and so $b_{2}=a_{4}$. There are two minimal nonfaces within $\sigma\cup\{a_{4}\}$. Let us use $\kappa_{2}=\{a_{4},c_{2},a_{3}\}$. Thus $W_{2}=\{1,2,3,4\}$ and the loop ends with k=2. Constructing ${\mathcal{E}}_{2}$ involves a choice since actions $c_{2}$ and $a_{3}$ each have action $a_{4}\mskip-1.5mu$’s source as a target. Let us pick ${\mathcal{E}}_{2}=\{a_{3}\}$.

Corollary 37 on page 37 arranges the expansive sets of actions in the order ${\mathcal{E}}_{2},{\mathcal{E}}_{1}$. Indeed, both $\,a_{3},a_{2},e_{3}\,$ and $\,a_{3},e_{3},a_{2}\,$ are informative action release sequences for $G$. Notice that action $e_{3}$ implies action $a_{3}$ in relation $A$. Consequently, the order ${\mathcal{E}}_{1},{\mathcal{E}}_{2}$ would *not * be acceptable.

Comment:  We can lengthen $\,a_{3},a_{2},e_{3}\,$ to the iars $\,c_{2},a_{3},a_{2},e_{3}$.  In fact, 12 of the 24 possible permutations of all the actions in $\mskip 0.5mu\sigma$ constitute informative action release sequences for $G$.

6 Counterexamples for Mixed Graphs

The assertion of Theorem 4 on page 4 need not hold for graphs containing a mix of deterministic, nondeterministic, and stochastic actions. Of course, there are many settings in which the assertion does hold. For instance, if one can find a hierarchical cyclic subgraph with the same state space as the given graph, then one can again prove the theorem, even if the graph contains stochastic actions. All the proof needs is for the hierarchical cyclic subgraph to be composed only of deterministic and nondeterministic actions. Absent such structure, it is very easy to construct a counterexample involving a mix of deterministic, nondeterministic, and stochastic actions. With some added effort, one may also construct counterexamples involving maximal strategies that attain singleton goals. This section presents such counterexamples.

6.1 A Counterexample with a Large Goal Set

Consider the graph and action relation of Figure 37. The graph contains two actions that have identical action edges, but differ in that one action is stochastic and the other action is nondeterministic. The stochastic action is $c_{1}=1\rightarrow p\mskip 1.25mu\{2,3,4\}$, for some probability distribution $p$ ascribing nonzero probabilities to each target, and the nondeterministic action is $a_{1}=1\rightarrow\{2,3,4\}$. Additionally, the graph contains three deterministic action, $b_{i}=i\rightarrow 1$, for $i=2,3,4$.

The graph is fully controllable based just on the set of actions $\{c_{1},b_{2},b_{3},b_{4}\}$, as one can see from the action relation or as follows: The system can attain state #1 from any other state by using strategy $\{b_{2},b_{3},b_{4}\}$. The system can reach a desired state in the set $\{2,3,4\}$ by repeatedly trying to do so using the stochastic action $c_{1}$, cycling back to state #1 if that action transitions to the wrong state. For instance, strategy $\{c_{1},b_{2},b_{3}\}$ will converge to state #4.

None of the actions $\{b_{i}\}$ can be in a strategy together with action $a_{1}$, but action $c_{1}$ can be. In fact, $\sigma_{234}=\{c_{1},a_{1}\}$ is a maximal strategy. The longest informative action release sequence contained in $\sigma_{234}$ is the strategy itself, revealed in the order $\,c_{1},a_{1}$. That iars has length 2, which is less than the number 3 demanded by Theorem 4.

One may easily generalize this example to graphs with $n$ states, for $n>4$, such that some maximal strategy consists of only two actions and therefore has an iars of length at most 2.

Key to this example is the oddity of having two nearly identical actions, with the only difference being that one action is stochastic and the other is nondeterministic. The graph would not be fully controllable with just the nondeterministic action. The stochastic action is needed to “sample with replacement”, i.e., “try and try again, until success”.

From a worst-case perspective, the strategy consisting of the nearly identical stochastic and nondeterministic actions amounts to no more than the nondeterministic action itself. So, why even include the nondeterministic action in the graph?

The answer is that it is a choice a system may make. Executing the stochastic action may entail greater cost than executing the nondeterministic action, because the nondeterministic action relieves the system of guaranteeing stochastic behavior. That may be desirable in some settings. At first it seems hardly so, because the only goal set one can attain using any strategy containing the nondeterministic action is a very large set (consisting of $n-1$ states in the generalized version). However, not caring about precise transitions is sensible when the graph is part of a larger graph and it does not matter what state the system passes through as a subgoal while attaining some overall goal. The next subsection explores such graphs further.

6.2 A Counterexample with a Small Goal Set

Previously, we saw the basis for a family of counterexamples in which the graph has $n$ states but contains a maximal strategy consisting of two actions with a goal set of size $n-1$. One might therefore hypothesize that Theorem 4 should merely assert the existence of an informative action release sequence of length $n-k$, with $k$ being the size of the goal set. In fact, such a theorem would also be false, since one can construct counterexamples to Theorem 4 using strategies that have goal sets of size 1, as this subsection demonstrates.

In constructing such counterexamples, one may take the fully controllable graph on $n$ states of the previous subsection and glue the set $\{2,\ldots,n\}$ to another graph, permitting direct motion from any state in $\{2,\ldots,n\}$ to a new state, #$(n+1)$. In order to retain full controllability, the system also needs to be able to move back from state #$(n+1)$. In this subsection, we use a single nondeterministic action. In the next subsection, will will use $n-1$ different nondeterministic actions. Numerous variations exist.

Figure 38 replicates a counterexample taken from [6], showing a graph and its action relation. Maximal strategy $\sigma_{5}=\{a_{1},d_{2},d_{3},d_{4},c_{1}\}$ converges to singleton goal state #5. The strategy contains 5 actions, but the longest informative action release sequences contained in $\sigma_{5}$ have length 3, which is less than the 4 demanded by Theorem 4. The reason no longer iars exists is because any one of the “downward” actions in the set $\{d_{2},d_{3},d_{4}\}$ implies the other two. That fact is clear from the action relation, but can also be understood as follows: First, knowing that a strategy contains one of the actions $d_{2}$, $d_{3}$, or $d_{4}$ means the strategy cannot contain the nondeterministic action $b_{5}$. Second, for a maximal strategy, not containing $b_{5}$ means the strategy must contain the entire set $\{d_{2},d_{3},d_{4}\}$.

6.3 A Counterexample with a Small Goal Set and Nonequivalent Inferences

Finally, we construct a counterexample similar to that of Figure 38, but without requiring equivalence between the deterministic downward actions flowing into state #$(n+1)$. Instead, any *two * of these downward actions will imply all the downward actions. One may achieve this inference by replacing the single nondeterministic action at state #$(n+1)$ with $n-1$ different nondeterministic actions. Each of these actions now has a target set of size $n-2$ contained within the set $\{2,\ldots,n\}$. With this counterexample in mind, one may see yet another infinite family of counterexamples, parameterized now by the number of downward actions $\{d_{i}\}_{i=2}^{n}$ that may be released before all are implied (with $n$ sufficiently large).

Figure 39 shows a graph $G$ in three panels. Figure 40 shows $G$’s action relation. There are six states. Four of the graph’s deterministic actions, namely $d_{2}$, $d_{3}$, $d_{4}$, $d_{5}$, transition to state #6, while four nondeterministic actions, $e_{2}$, $e_{3}$, $e_{4}$, $e_{5}$, transition away from state #6. Each of those nondeterministic actions has a target set of size three that is a subset of $\{2,3,4,5\}$. The following table shows which pairings of $d_{i}$ and $e_{j}$ actions create minimal nonfaces in $\Delta_{G}$. One sees that any single action drawn from $\{d_{2},d_{3},d_{4},d_{5}\}$ is potentially consistent with strategies not involving any other $d_{i}$ action, but that any two of the $\{d_{i}\}$ actions imply them all. (Any two of the $\{d_{i}\}$ eliminate all $\{e_{j}\}$, so the given maximal strategy must contain all $\{d_{i}\}$.)

[TABLE]

Maximal strategy $\sigma\mskip-4.0mu=\mskip-4.0mu\{a_{1},d_{2},d_{3},d_{4},d_{5},c_{1}\}$ converges to goal state #6 and contains $\mskip-0.05mu6\mskip-0.75mu$ actions. However, since at most two of the actions $\{d_{i}\}$ are informative, the longest informative action release sequences contained in $\sigma$ have length 4. That is less than the 5 demanded by Theorem 4.

Acknowledgments

Many thanks to Rob Ghrist, Steve LaValle, Ben Mann, and Matt Mason, for advice and support related to this research and its earlier foundations within SToMP.