Random Networks, Graphical Models, and Exchangeability
Steffen Lauritzen, Alessandro Rinaldo, Kayvan Sadeghi
TL;DR
This paper explores the structure of exchangeable random network models, characterizing their Markov properties, and identifying classes of models based on subgraph densities, degree distributions, and consistency under subnetworks.
Contribution
It provides a comprehensive characterization of Markov structures for finitely exchangeable networks and introduces new classes of models linked to bidirected Kneser graphs.
Findings
Empirical subgraph densities are maximum likelihood estimates.
Identified a new class of Markov network models for exchangeable graphs.
Only certain graph structures are consistent under subnetwork formation.
Abstract
We study conditional independence relationships for random networks and their interplay with exchangeability. We show that, for finitely exchangeable network models, the empirical subgraph densities are maximum likelihood estimates of their theoretical counterparts. We then characterize all possible Markov structures for finitely exchangeable random graphs, thereby identifying a new class of Markov network models corresponding to bidirected Kneser graphs. In particular, we demonstrate that the fundamental property of dissociatedness corresponds to a Markov property for exchangeable networks described by bidirected line graphs. Finally we study those exchangeable models that are also summarized in the sense that the probability of a network only depends onthe degree distribution, and identify a class of models that is dual to the Markov graphs of Frank and Strauss (1986). Particular…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14
Figure 15
Figure 16
Figure 17
Figure 18
Figure 19
Figure 20Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStochastic processes and statistical mechanics · Markov Chains and Monte Carlo Methods · Complex Network Analysis Techniques
Random Networks, Graphical Models,
and Exchangeability
Steffen Lauritzen
University of Copenhagen Steffen Lauritzen, Department of Mathematical Sciences, University of Copenhagen, Universitetsparken 5, 2100, Copenhagen, Denmark. email: [email protected]
Alessandro Rinaldo
Carnegie Mellon University
Kayvan Sadeghi
University of Cambridge
Abstract
We study conditional independence relationships for random networks and their interplay with exchangeability. We show that, for finitely exchangeable network models, the empirical subgraph densities are maximum likelihood estimates of their theoretical counterparts. We then characterize all possible Markov structures for finitely exchangeable random graphs, thereby identifying a new class of Markov network models corresponding to bidirected Kneser graphs. In particular, we demonstrate that the fundamental property of dissociatedness corresponds to a Markov property for exchangeable networks described by bidirected line graphs. Finally we study those exchangeable models that are also summarized in the sense that the probability of a network only depends on the degree distribution, and identify a class of models that is dual to the Markov graphs of Frank and Strauss (1986). Particular emphasis is placed on studying consistency properties of network models under the process of forming subnetworks and we show that the only consistent systems of Markov properties correspond to the empty graph, the bidirected line graph of the complete graph, and the complete graph.
Keywords: bidirected Markov property, de Finetti’s theorem, exchangeable arrays, exponential random graph model, graph limit, graphon, Kneser graph, marginal beta model, Möbius parametrization, Petersen graph.
1 Introduction
Over the last decades, the collection and rapid diffusion of network data originating from a wide spectrum of scientific areas have created the need for new statistical theories and methodologies for modeling and analyzing large random graphs. There now exists a very large and rapidly growing body of literature on network analysis: see, for example, Kolaczyk (2009), Newman (2010), and references therein. Since the seminal contributions of Frank and Strauss (1986) and Holland and Leinhardt (1981), the field of statistical network modeling has advanced significantly and researchers have now gained a much broader understanding of the properties of network data and the many open challenges associated to network modeling.
A common feature shared by many network models is that of invariance to the relabeling of the network units, or (finite) exchangeability, whereby isomorphic graphs have the same probabilities, and are therefore regarded as statistically equivalent. Exchangeability is a basic form of probabilistic invariance, but also a natural and convenient simplifying assumption to impose when formalizing statistical models for random graphs. Examples of popular network models which rely on exchangeability include many exponential random graph models, the stochastic block model, graphon-based models, latent space models, to name a few.
In the probability literature, exchangeability of infinite binary arrays — which encompasses networks — is very well understood: see for example Aldous (1981, 1985); Lauritzen (2003, 2008); and Kallenberg (2005). Of particular relevance here is the work of Diaconis and Janson (2008) (but see also Orbantz and Roy, 2015), which details the connections between exchangeability of random graphs and the notion of graph limits developed in Lovász and Szegedy (2006). While the existing results provide a canonical, analytic representation of infinite exchangeable networks (see later in Section 2), they appear to be too implicit for statistical modeling purposes. Indeed, the properties implied by such representation do not directly yield simple parametric statistical models for finite networks. Furthermore, while an exchangeable probability distribution for an infinite networks by construction induce consistent families of exchangeable distributions for all its finite subnetworks, there is no guarantee that an exchangeable distribution for networks of a given size may be extendable to a probability distribution for larger (possibly infinite) networks, or that the induced distributions share the similar properties. Thus, a priori any statistical model for finite exchangeable networks that does not conform to the analytic representation of infinite exchangeable networks need not be compatible in any meaningful way with models of the same type over networks of different sizes. These issues are of great significance, as they render very difficult to determine the extent to which statistical inference based on a subnetwork applies to a larger network, as argued in Crane and Dempsey (2015), and Shalizi and Rinaldo (2013).
The main goal of our article is to reveal and point out similarities and connections between seemingly unrelated concepts from different parts of the literature — in particular graphical models, network models, and notions of exchangeability — and our analysis is attempting to enhance understanding of the models and their properties.
We rely on the theory of exponential families and, especially, of graphical models (Lauritzen, 1996) in order to categorize Markov properties implied by the assumption of exchangeability, and the statistical models that can be derived from it. The idea of modeling networks as Markov random fields on binary variables is not new: it was originally suggested by Frank and Strauss (1986), who argue that a particular class of exponential random graph models (to be discussed below in Section 4) is able to capture what the authors, somewhat arbitrarily, posit as a natural form of dependence structure for networks, as well as the symmetries implied by exchangeability. In contrast, our analysis of the Markov properties of exchangeable network models is both principled, as it only relies on the assumption of exchangeability, and exhaustive, as it produces a complete list of all Markov properties expressed by exchangeable network models. We make the following specific contributions:
- •
We describe an exponential family representation for exchangeable network models where the sufficient statistics are given by all subgraph counts. The subgraph frequencies are maximum likelihood estimators of the corresponding mean value parameters, which are the marginal probabilities of all subgraphs.
- •
We study the Markov properties of finitely exchangeable network models, and show that such models are compatible only with four types of non-trivial conditional independence structures.
- •
We demonstrate that statistical models for finite networks induced by extremal exchangeable distributions on an infinite network are Markov with respect to a distinguished bidirected graphical model for marginal independence.
- •
Finally, we propose a novel class of models for exchangeable networks that are also summarized, in the sense that their properties only depend on the degree distribution of the observed network.
We should emphasize that the scope of this paper has been strictly limited to the study of full exchangeablity and Markov structures in the sense used for standard graphical models and, as we show, the assumption of full exchangeabiliy, extendibility, and Markov properties in this sense is very restrictive.
There are important types of Markov properties for random networks that we have not considered in this paper, notably that of partial conditional independence (Snijders et al., 2006), also used in Hunter et al. (2008). This concept allows the conditional independence properties of specific ties in a network to depend on the configuration of the remaining network. This type of Markov property is common in the analysis of spatial point processes, where it is referred to as data dependent Markov neighbourhoods (Baddeley and Møller, 1989); indeed, a random network can be considered as a point process on the space of possible ties. Also in the literature on Bayesian networks and graphical models, an analogous Markov theory has evolved and here the term context-specific independence is often used, see e.g. Boutilier et al. (1996) or Nyman et al. (2014) for a recent discussion.
Similarly, it is of interest but outside the scope of the present paper to discuss other variants of exchangeability, where we in particular mention partial exchangeability or block exchangeability as used in Schweinberger and Handcock (2015) or models derived from exchangeable measures, as in Caron and Fox (2017).
The structure of our paper is as follows. In Section 2 we develop the necessary terminology and detail the concepts of exchangeable arrays, random networks, bidirected Markov properties, and important parametrizations of the models. In Section 3 we provide parametrizations for exchangeable networks and their maximum likelihood estimation. In Section 4 we investigate the consequences of exchangeability in its interplay with associated Markov properties. In Section 5 we study the consequences of adding the restrictions of summarizedness, and in Section 6 we study consistency properties under subsampling of various systems of network models.
2 Preliminaries
In this section we gather some background material on graph-theoretic concepts, parametrizations of distributions of binary arrays and graphical modeling.
2.1 Graph-theoretic concepts
Following West (2001), a labeled graph is determined by an ordered pair consisting of a vertex set , an edge set , and a relation that with each edge associates two vertices, called its endpoints. When vertices and are the endpoints of an edge, these are adjacent and we write ; we denote the corresponding edge as .
In this paper, we will restrict to simple graphs, i.e. graphs without loops (the endpoints of each edge are distinct) or multiple edges (each pair of vertices are the endpoints of at most one edge); simple graphs are determined by the pair alone. We will distinguish three types of edge, denoted by arrows, arcs (solid lines with two-headed arrows) and lines (solid lines). Arrows can be represented by ordered pairs of vertices, while arcs and lines by -subsets of the vertex set. The graphs in the present paper only contain one of these types, and are called, respectively, directed, bidirected, or undirected.
The labeled graphs and are considered equal if and only if . We omit the term labeled when the context prevents ambiguity.
A subgraph of a graph is a graph such that and and the assignment of endpoints to edges in is the same as in . We will write to signify that is a subgraph of . For a graph , any non-empty subset of the vertices generates the subgraph consisting of all and only vertices in and edges between two vertices in , called the subgraph induced by . Similarly, a subset of edges induces a subgraph that contains the edges in and all and only vertices that are endpoints of edges in . We note that *edge induced subgraphs have no isolated vertices. *
A complete graph is a graph where all nodes are adjacent, and a -star is a graph where one node (called the hub) is adjacent to all other nodes and there is no other edge in the graph. The line graph of a graph is the intersection graph of the edge set , i.e. its vertex set is and if and only if and have a common endpoint (West, 2001, p. 168). We will in particular be interested in the line graph of the complete graph, which we will refer to as the incidence graph.
2.2 Homomorphism densities
We will rely on the notion of subgraph homomorphism density, a graph-theoretic concept that is central to the theory of graph convergence (see, e.g., Lovász, 2012).
A map between the vertex sets of two simple graphs and is a homomorphism if it preserves edges, i.e. if in implies . A homomorphism is an isomorphism if it is a bijection and its inverse is a homomorphism, i.e. also preserves non-edges. If there is an isomorphism between and , the graphs are isomorphic, and we write . An isomorphism can also be thought of as a relabelling of the vertex set. The isomorphism relation is an equivalence relation among graphs with the same vertex set. Accordingly, for a graph we may identify the corresponding equivalence class as an unlabeled graph with vertices. An isomorphism between the graphs and is an automorphism. Then denotes the size of the automorphism group , i.e. the group of automorphisms of .
Following Lovász (2012, Ch. 5.2) and assuming that , we let denote the number of injective graph homomorphisms between and . Note that if is a complete graph (i.e. all the vertices are adjacent), all maps are homomorphisms, and . Furthermore, we have that . The injective homomorphism density of in is defined to be
[TABLE]
the proportion of all injective mappings from into that are homomorphisms. It is immediate from the above definition that homomorphism densities are invariant under isomorphisms, in the sense that , for any pairs and . Thus, density homomorphisms are also well defined over unlabeled graphs. Indeed, the injective homomorphism density can be interpreted as the probability that is realized as a random subgraph of (see, e.g. Diaconis and Janson, 2008).
Finally, we will be interested in the number of (not necessarily induced) subgraphs of that are isomorphic to . This is given as
[TABLE]
Clearly, also remains unchanged if and are replaced by isomorphic graphs and so that is well defined over unlabeled graphs.
2.3 The Möbius parametrization for binary
arrays
As explained later in detail, network modeling can be considered equivalent to modeling the joint distribution of a collection of binary random variables. For this purpose we will find it convenient to rely on a particular parametrization of such distributions that is based on the Möbius transform (see, e.g. Lauritzen, 1996, Appendix A3) and that therefore, following Drton and Richardson (2008), we call the Möbius parametrization.
Let be a finite set indexing a collection of binary random variables taking value in and for a non-empty subset of let . The Möbius parametrization arises from taking the Möbius transform of the vector of joint probabilities of each point in . This transformation, which is linear and invertible, in turns yields a vector of marginal, as opposed to joint, probabilities. In detail, let denotes the set of all subsets of and, for each non-empty , set , where is the -dimensional vector of all ’s. Further, set . Then, using Möbius inversion, we have that, for each ,
[TABLE]
where . Similarly, for each ,
[TABLE]
The marginal probabilities are called the Möbius parameters of the distribution of . The above formulae show that there is a one-to-one and linear relation between joint probabilities and their Möbius parameters and that the only restriction on the set of feasible Möbius parameters is that all expressions of the form (2) should be non-negative, and .
We can represent the set of all strictly positive probabilities on as an exponential family with canonical sufficient statistic given by
[TABLE]
where , with
[TABLE]
see, for instance, Frank and Strauss (1986). The exponential family on generated by these sufficient statistics consists of all probability distributions of the form
[TABLE]
for any choice of canonical parameters , and where is the log-partition function, ensuring that probabilities add to unity. The above exponential family is minimal, full and regular, and the mean value parameters (Barndorff-Nielsen, 1978) for this representation are precisely the Möbius parameters corresponding to non-empty subsets of :
[TABLE]
2.4 Undirected and bidirected graphical models
Graphical models (see, e.g. Lauritzen, 1996) are statistical models expressing conditional independence clauses among a collection of random variables indexed by a finite set . A graphical model is determined by a graph over the indexing set , and the edge set (which may include edges of undirected, directed or bidirected type) encodes conditional independence relations among the variables, or Markov properties. A joint probability distribution for is Markov with respect to if it satisfies the Markov properties expressed by .
For non-empty subsets and of and a (possibly empty) subset of , we use the notation A\mbox{>\perp\perp }B\,|\,S as shorthand for the conditional independence relation X_{A}\mbox{>\perp\perp }X_{B}\,|\,X_{S}, thus identifying random variables with their labels. When the independence relation is intended as marginal independence.
For an undirected graph — where all edges are undirected — the (global) Markov property expresses that A\mbox{>\perp\perp }B\,|\,S when every path between and has a vertex in or, in other words, separates from in .
We shall be specifically interested in graphical models given by a bidirected graph where all edges are bidirected. For such graphs the (global) Markov property (Cox and Wermuth, 1993; Kauermann, 1996; Richardson and Spirtes, 2002; Richardson, 2003) expresses that
A\mbox{>\perp\perp }B\,|\,S when every path between and has a vertex outside , i.e. separates from . Note the obvious duality between this and the Markov property for undirected graphs.
For example, in the undirected graph of Figure 1(a), the global Markov property implies that \{i,l\}\mbox{>\perp\perp }k\,|\,j , whereas in the bidirected graph of Figure 1(b), the global Markov property implies that \{i,l\}\mbox{>\perp\perp }k. Notice that, for simplicity, we write and instead of and as these are single elements.
For a system of random variables we define the dependence skeleton of , denoted by to be the undirected graph with vertex set such that vertices and are not adjacent if and only if there is some subset of so that u\mbox{\,\perp!!!\perp\,}v\,|\,S. Thus, if is Markov with respect to an undirected graph — such as the case considered by Frank and Strauss (1986), who chose to be the set of remaining variables — would be a subgraph of ; whereas if is Markov with respect to a bidirected graph, we may choose and the corresponding skeleton will be a subgraph of the graph obtained by replacing all arcs with lines.
Bidirected graphical models for binary variables
For the bidirected case, the exponential representation does not simplify. However, in the Möbius parametrization the bidirected Markov property takes a particular simple form. Indeed Drton and Richardson (2008) showed that a distribution is Markov with respect to the bidirected graph if and only if for any that is disconnected in , the corresponding Möbius parameter satisfies
[TABLE]
where partitions into inclusion maximal connected sets (with respect to ). Thus, if we let be the set of all non-empty connected subsets of (with respect to ), the bidirected binary graphical model with graph is completely, injectively, and smoothly parametrized by the Möbius parameters . More precisely we have that
[TABLE]
where denotes the collection of inclusion maximal connected subsets of . Further, the Möbius parameters vary freely save for the restrictions that and the right-hand expressions in (2) and (5) must be non-negative for all choices of . For example, for the bidirected graph of Figure 2 we have that
[TABLE]
Drton and Richardson (2008) further show that the family of positive bidirected Markov distributions becomes a curved exponential family due to the restriction (4) which is non-linear in the canonical parameters and log-linear in the mean value parameters (Roverato et al., 2013). Finally, the dimension of the corresponding model is the cardinality of , the number of connected induced subgraphs of .
3 Network models and exchangeability
3.1 Some terminology
Given a finite or countably infinite node set — representing individuals or actors in a given population of interest — we define a random network over to be a collection of binary random variables taking values [math] and indexed by a set of dyads. Throughout the paper we take to be the set of all unordered pairs of nodes in , and nodes and are said to have a tie if the random variable takes the value , and no tie otherwise. Thus, a network is a random variable taking value in and can therefore be seen as a random simple, undirected, labeled graph with node set , whereby the ties form the random edges of the graphs. We will write for the set of all simple, labeled undirected graphs on .
We use the terms network, node, and tie rather than (random) graph, vertex, and (random) edge to differentiate from the terminology used in the graphical model sense. Indeed, as we shall discuss graphical models for networks, we will also consider each dyad as a vertex in a graph representing the dependence structure of the random variables associated with the dyads.
3.2 Exponential random graph models
Exponential random graph models (Frank, 1991; Wasserman and Pattison, 1996), or ERGMs in short, are among the most important and popular statistical models for network data and, as we will see, the set of exchangeable distributions form an ERGM.
ERGMs are exponential families of distributions (Barndorff-Nielsen, 1978) on whereby the probability of observing a network can be expressed as
[TABLE]
where are canonical sufficient statistics which capture some important feature of , is a point in the canonical parameter space , and is the log-partition function; in (6) we have thus chosen counting measure as the base measure of the representation.
The choice of the canonical sufficient statistics and any restriction imposed on will determine the properties of the corresponding ERGM. The simplest ERGM is the Erdös–Rényi model (Erdös and Rényi, 1960), where there is only one parameter , and the canonical sufficient statistic is the number of ties in . This is equivalent to ties occurring independently with probability and .
Beta models are also ERGMs and they can be considered a generalization of the Erdös-Rényi model: in these models it is also assumed that ties occur independently; the probability of a tie between nodes and is given as:
[TABLE]
where can be interpreted as parameters that determine the propensity of node to have edges. The probability of a network is thus
[TABLE]
where is the degree sequence of , i.e. is the number of ties in involving node .
Other ERGMs include, for example, the exponential family models of Holland and Leinhardt (1981) and their modifications (Fienberg and Wasserman, 1981), as well as the Markov graphs of Frank and Strauss (1986).
If , any probability distribution for a network induces a distribution for the subnetwork corresponding to the dyads in . In particular, is the subgraph of induced by . One problem with ERGMs is that, typically, needs not be the related to in a meaningful way, and needs not be an ERGM itself. In that sense, the ERGMs are not marginalizable; see also Snijders (2010) as well as Shalizi and Rinaldo (2013). The Erdös–Rényi models and the beta models are marginalizable in this sense. Many other network models suffer from this issue: see, e.g., Crane and Dempsey (2015). We shall return to this and related issues in more detail in Section 6.
3.3 Parametrizations of random network
models
Since networks are arrays of binary variables, we may use the Möbius parametrization and the exponential form in Section 2.3 to represent their distributions.
For a given finite node set , we thus let be the collection of all subsets of . Now if is a network on and a non-empty set in , then is the subnetwork indexed by the dyads in . It will be convenient to identify each non-empty set with the edge-induced subgraph of the complete graph on comprised of the edges in . Equivalently, represents all subgraphs of the complete graph on without isolated nodes. In particular, for a non-empty , we may equivalently write the event as the event that the graph is a subgraph of the random graph .
The Möbius parameters corresponding to the distribution of a network are , where
[TABLE]
Thus, the Möbius parameters for a network distribution are just the implied probabilities of observing all subgraphs of the complete graph on without isolated nodes. In the exponential family parametrization (3), the canonical sufficient statistic is defined over graphs , and the coordinate indexed by any non-empty is simply the indicator function
[TABLE]
Example 1**.**
Suppose that we observe the network in Figure 3 (a), where .
For any probability measure on , we have that, under both the Möbius parametrization and the canonical parametrization in (3),
[TABLE]
where is the unique -dimensional vector parametrizing the distribution of . Notice that the Möbius parametrization is considerably simpler than the exponential family parametrization in this instance, since is a dense network and only parameters corresponding to supergraphs of the observed network enter into the calculation. However, if we instead observe the sparser network in Figure 3 (b), we get
[TABLE]
Thus the exponential form is simpler for the sparse case, save for the fact that the log-partition function is complicated.
3.4 Exchangeability
We are concerned with probability distributions on networks that are exchangeable: invariant under relabelings of the node set . Exchangeability is a well-known form of probabilistic invariance, and, from a modeling perspective, also a natural simplifying assumption to impose when formaliziing statistical models for networks. Below we summarize well-known facts on exchangeability.
A distribution of a random array over a finite nodeset is said to be weakly exchangeable (WE) (Silverman, 1976; Eagleson and Weber, 1978) if for all permutations we have that
[TABLE]
If the array is symmetric — i.e. , we say it is symmetric weakly exchangeable (SWE). In the following we shall for brevity say that is WE, SWE, etc. in the meaning that its distribution is.
A symmetric binary array with zero diagonal can be interpreted as a matrix of ties (adjacency matrix) of a random network and thus the above concepts can be translated into networks. A random network is exchangeable if its adjacency matrix is SWE. Then a random network is exchangeable if and only if its distribution is invariant under relabeling of the nodes of the network. We will distinguish the case of a finite node set from that of a (countably) infinite one, and in the former case we will speak of finite exchangeability. A random network over a countably infinite nodeset is said to be exchangeable, if every finite induced subnetwork for is.
When is infinite, exchangeability is well-understood. Indeed, any doubly infinite random SWE array can be represented as a mixture of so-called -matrices (Diaconis and Freedman, 1981) in the following way. Let be the convex set of all exchangeable distributions for networks with a countably infinite number of nodes and let denote the extreme points of that set. Then we have:
Proposition 1**.**
* if and only if for every finite ,*
[TABLE]
where is the induced probability on and is a measurable function from to . The function is unique up to a measure-preserving transformation of the unit interval.
In a graph theoretic context, (equivalence classes of) -matrices are also known as graphons (Lovász and Szegedy, 2006).
Elements of as given in (12) are all dissociated (Silverman, 1976) in the sense that are independent of whenever in fact it holds (Aldous, 1981, 1985) that
Proposition 2**.**
* if and only if is exchangeable and dissociated.*
We emphasize that the representation of exchangeable distributions on networks by means of -matrices requires an infinite node set . If is finite, a finitely exchangeable network on needs not have a representation as a mixture of -matrices, and need not be extendable, i.e. the induced probability measure from a probability distribution over networks on a larger node set. In fact, the properties of finitely exchangeable networks are distinctively different; see for example Diaconis (1977); Diaconis and Freedman (1980); Matúš (1995); Kerns and Székely (2006); and Konstantopoulos and Yuan (2015). We will return to the issue of extendability later in Section 6 and focus instead on the properties of finitely exchangeable network distributions.
3.5 Exponential representation of exchangeable networks
For exchangeable random networks on a finite nodeset , both the Möbius and the exponential parameters simplify. More precisely, is an exchangeable distribution on if and only if whenever , i.e. whenever and are identical save for a permutation of their labels. Thus, it follows that a random network is exchangeable if and only if whenever , where .
Next we let denote the set of all unlabeled graphs on and write for the empty graph. Each can be identified with an isomorphism class in , which we will denote with . Then, any exchangeable distribution on is parametrized by , with , where , for any . Collecting identical terms in (2) we obtain
[TABLE]
where is the number of edges in and is the number of graphs in that contain as a subgraph.
Similarly, it holds for the exponential representation (3) that a probability distribution on is exchangeable if and only if for all and in with , the corresponding canonical parameters satisfy . Thus we can represent the family of exchangeable network distributions on as the exponential family of probability distributions of the form
[TABLE]
for any choice of the canonical parameters , where is the log-partition function and
[TABLE]
is the number of graphs in the isomorphism class corresponding to that are subgraphs of . Indeed from (1) we have that
[TABLE]
Note that the set of exchangeable distributions again form a linear and regular exponential family with canonical sufficient statistics , canonical parameters , and mean value parameters
[TABLE]
for . In other words, the family of finitely exchangeable network distributions is an ERGM.
By standard theory of exponential families (Barndorff-Nielsen, 1978), upon observing a network , the maximum likelihood estimate of the Möbius parameters under the assumption of exchangeability is obtained by equating the observed canonical statistic to its expectation. As a result, the MLE of , for any non-empty is
[TABLE]
Strictly speaking, this is not the maximum likelihood estimate in the exponential family, but rather its completion (Barndorff-Nielsen, 1978) obtained by including all limit points of the set of mean value parameters. In fact, the canonical parameters are not estimable based on observation of a single network. Note that, interestingly, the maximum likelihood estimates of the Möbius parameters are exactly the injective homomorphism densities introduced in Section 2. For an extreme and infinitely exchangeable network (the dissociated case), these estimates are also well known to be consistent (Lovász and Szegedy, 2006, Theorem 2.5).
Example 2**.**
Consider the networks and in Figure 3. Using the exponential representation (14) we see — with a notation that hopefully is transparent — that the probability of observing is
[TABLE]
In terms of the Möbius parametrization the expression simplifies to
[TABLE]
If is observed, the non-zero estimates of Möbius parameters for are
[TABLE]
These numbers are obtained from (17) by calculating, for example, as
[TABLE]
Note that the canonical parameters in (18) are not estimable in this case as the set of sufficient statistics are at the boundary of their convex support.
4 Independence structures for exchangeable
networks
In this section we will investigate the Markov properties of exchangeable network distributions on a finite node set.
4.1 Possible independence structures
Recall that we use the term incidence graph for the line graph of a complete graph on , whereby has edges between dyads which are incident i.e. dyads having a node in common. Figure 4 displays the incidence undirected graph for and a bidirected version of the same.
In fact, we show next that the skeleton graph of a finitely exchangeable network distribution can only be one of four possible types.
Proposition 3**.**
The dependence skeleton of an exchangeable random network is one of the following:
- (a)
the empty graph; 2. (b)
the incidence graph ; 3. (c)
the complement of the incidence graph ; 4. (d)
the complete graph.
Proof.
Let . Exchangeability implies that one can relabel the vertices of the dependence skeleton corresponding to every permutation of without changing independence statements of form i\mbox{>\perp\perp }j\,|\,S for some . If the dependence skeleton is empty or complete it is clear that relabeling would not change such independence statements.
If there is an edge , for mutually non-equal , and a missing edge , for mutually non-equal , in the dependence skeleton, then by the relabeling corresponding to sending to we obtain a missing edge between and . However, in the latter labeling, there is an independence of form ij\mbox{>\perp\perp }kl\,|\,S whereas in the former labeling there is no independence of form ij\mbox{>\perp\perp }kl\,|\,S. Hence, such a graph cannot be the dependence skeleton of an exchangeable random network.
Similarly, if there is an edge , for mutually non-equal , and a missing edge , for mutually non-equal , in the dependence skeleton, then by the relabeling corresponding to sending to we obtain a missing edge between and . With the same argument as in the previous case, such a graph cannot be the dependence skeleton of an exchangeable random network.
The remaining cases are when either all the pairs of vertices are adjacent and all the pairs of vertices are not adjacent, which leads to the ; or all the pairs of vertices are adjacent and all the pairs of vertices are not adjacent, which leads to .
∎
To give a complete characterization of all possible graphical Markov structures with these skeletons, we need to give a precise definition of a generic graphical Markov structure. We abstain from doing so but just mention that it is clear that directed edges are not relevant for exchangeable random networks. If we vary the type of edge so that edges are either all bidirected or undirected, this leads to six different dependence structures since the undirected and bidirected graphs in cases (a) and (d) are Markov equivalent. If an exchangeable random network satisfies the Markov property associated with an undirected or bidirected dependence graph with vertex set then is one of these cases. Dependence structures corresponding to the complete graph in (d) are uninteresting. The case (a) corresponds to the Erdös–Renyi model. Structures corresponding to the other four dependence graphs are classified below, where our main focus is on the bidirected incidence graph case as it corresponds to the dissociated model and has certain desired properties concerning extendability; see Section 6.
Undirected incidence graph – the Frank-Strauss model.
Frank and Strauss (1986) showed the result below:
Proposition 4**.**
Every clique of the incidence graph of the complete network base corresponds exactly to a subnetwork of that is either a triangle or a -star.
Based on this, they show that a random network is Markov with respect to if and only if the canonical parameters are zero unless is a -star or a triangle. Assuming also exchangeability, this model is known as the Frank-Strauss model and studied extensively in Frank and Strauss (1986).
Example 3**.**
Consider again the network in Figure 3 and any choice of the canonical parameters . Following Example 2, the probability of observing under the Frank–Strauss model becomes
[TABLE]
whereas no further simplification appears in terms of the mean-value parameters .
Undirected complement of the incidence graph.
Cliques in the complement of the incidence graph correspond to collections of disjoint dyads. Therefore, the corresponding model is an ERGM with sufficient statistics corresponding to collections of disjoint dyads (including the subnetwork that is a single dyad). For the case of a network with five nodes, the dependence graph is the famous Petersen graph (Petersen, 1898) and for vertices, the graph is known as the Kneser graph (Godsil and Royle, 2001, Ch. 7); for this reason we shall refer to the model determined by all exchangeable random networks that are Markov w.r.t. the Kneser graph as the Kneser model.
Example 4**.**
Consider again the network in Figure 3. Still following Example 2 we get using the exponential representation (14) and the Markov property w.r.t. that in the Kneser model, the probability of observing is
[TABLE]
for any .
Bidirected complement of the incidence graph.
Every connected subset of the complement of the incidence graph corresponds to a disconnected subnetwork of the of size or the subnetwork that is a single tie (which corresponds to a single vertex in ). Therefore, the corresponding model has Möbius parameters corresponding to these. This model satisfies a property that is dual to dissociatedness so that, for example X_{ij}\mbox{\,\perp!!!\perp\,}X_{jk}.
Example 5**.**
If the random network is Markov w.r.t. the complement of the bidirected incidence graph , the Möbius parametrization of the probability of the network in Figure 3 is
[TABLE]
This has been obtained, for example, as z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-7.pdf}}}}}=(z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-1.pdf}}}}})^{2}z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}} since, for an arbitrary labeling of
, two of the three disconnected induced subgraphs , , and of correspond to the subnetwork
and one to
.
Later in Section 6 we will argue that the only Markov structures that are also extendable to arbitrarily large networks are the complete graph, the empty graph, and the bidirected incidence graph; hence the latter of these structures is the main focus of the present paper.
4.2 Bidirected incidence graph – the dissociated model
We now consider in detail the class of finitely exchangeable network distributions that are Markov with respect to the bidirected incidence graph , where is finite. This is equivalent to requiring an exchangeable network on to be dissociated: are independent of for each pair disjoint subsets and of .
We begin by stating a simple yet useful property of the incidence graph. For a given finite node set , let denote the subsets of dyads corresponding to connected subgraphs of the complete graph on .
Lemma 1**.**
A random network is dissociated if and only if for any non-empty subnetwork it holds that
[TABLE]
where the are the connected components of .
Proof.
The proof follows directly from (4) and the fact that there is a one-to-one correspondence between the set of all connected subgraphs of and .
∎
With some abuse of notation, for any non-empty , we will write for the decomposition of one arbitrary graph in the isomorphism class into the disjoint union of its connected components in . Note that each graph in has the same number of isomorphic connected components. By exchangeability and dissociatedness, for any ,
[TABLE]
regardless of the choice of the representing graph in . From (4) and (13) we thus arrive at the following result.
Proposition 5**.**
A network on is exchangeable and dissociated if and only if
[TABLE]
It is worth noting that the model combining dissociatedness and exhangeability remains a log-mean linear model in the sense of Roverato et al. (2013).
Example 6**.**
The additional restriction of dissociatedness does not yield any helpful simplification of the exponential representations in (18) from 2, since the dissociatedness conditions entails restriction on the possible values of the canonical parameters and not on the form of the probabilities. For the observed network , no further simplification of its Möbius parametrization in (19) is possible.
Continuing with Example 2, under dissociatedness, the likelihood function in (19) should now be maximized not only under the constraints that all expressions in (2) be non-negative, but also under the non-linear constraints described in (4), which in this case reduces to just one constraint z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}}=z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-1.pdf}}}}}^{2}. Thus, although the likelihood function is linear in , numerical maximization is in general necessary. In this particular case we find that
[TABLE]
and the remaining parameter estimates are zero, which should be compared to the results in the non-dissociated case. In fact, this estimate represents a mixture of the uniform distribution of all networks isomorphic to the observed network , and the empty network, with weights and , respectively.
We emphasize that when dissociatedness is assumed, the maximum likelihood estimator may no longer be unique, as the example below shows.
Example 7**.**
Suppose that we observe the network in Figure 5. Then the likelihood function assuming dissociatedness is maximized at any value of satisfying by the quantities
[TABLE]
and all other equal to zero. Indeed this corresponds to a random network that has probability of being isomorphic to the observation and the remaining probability mass of is distributed arbitrarily between a triangle plus an isolated point, and a 3-star.
5 Exchangeable and summarized random networks
An assumption commonly used in network modeling is that the probability of a network configuration in be a function of its degree distribution , where
[TABLE]
with the degree of node and if and [math] otherwise. Note that here the degree distribution is not normalized and is the number of nodes in with degree .
Lauritzen (2008) defines a random binary array to be weakly summarized (WS) if there is a function such that it holds that
[TABLE]
where and are the row and column sums. For symmetric arrays, it holds that , and a distribution with the above property may be called symmetric weakly summarized (SWS). Again, this can be translated into statements about networks: A random network on is summarized if, for any , , where is the degree of node . SWE distributions are generally not SWS or vice versa (Lauritzen, 2008). If a distribution is both SWE and SWS then we write that the distribution is SWES. When is infinite and is a SWES array whose distribution is in , Lauritzen (2008) showed that 1 holds with having a specific form: (Lauritzen, 2008). The beta model in (7) then emerges when further conditioning on and letting .
5.1 Characterizing summarizing statistics
We now study exchangeable and summarized distributions on a finite node set . Isomorphic graphs have the same degree distribution; but, as shown for instance in Example 8 below, the opposite is not true and thus the set of summarized and exchangeable network distributions on is a subfamily of the set of exchangeable distribution on . In order to characterize the types of restriction that summarizedness entails, we begin with some preliminary lemmas which identify which subgraph counts are functions of the degree distribution only. Recall that subgraph counts are canonical sufficient statistics for the exponential family of exchangeable network distributions: see Section 3.5.
Lemma 2**.**
Given an unlabeled graph , it holds that is a function of the degree distribution of , for all , if
- (a)
, a -star for any size or U=\hskip 3.50006pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}, the disjoint union of two independent edges, or 2. (b)
, the complete graph of size , or , the complete graph of size with one edge removed.
Proof.
To prove (a), note that a [math]-star is simply a vertex and a -star is an edge. If for then
[TABLE]
where the sum only effectively extends at most from to as all other terms are zero. If we have to divide by two, and obtain
[TABLE]
where is the number of edges in . The formula (26) is also known as Euler’s handshaking lemma; see e.g. (Wilson, 1996, p.12). Finally, if U=\hskip 3.50006pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}} we have
[TABLE]
For (b), we have that
[TABLE]
where is at the th entry. The top row corresponds to the complete graph, the second row to the complete graph with one edge removed, and the third row to a complete graph with a vertex from which edge is removed.
A similar function can be provided for the case of , but we abstain from giving the somewhat cumbersome details.
∎
The converse to the statement in Lemma 2 only holds with some modification:
Lemma 3**.**
Given an unlabeled graph , if is a function of the degree distribution of , for all , then
- (a)
if has at most vertices, is a -star or U=\hskip 3.50006pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}, the disjoint union of two independent edges; 2. (b)
if has at most vertices, is a -star, or U=\hskip 3.50006pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}, or , or , a complete graph on vertices with one edge removed.
Proof.
It is enough to show that for any that is not one of the above graphs, there exist two graphs and with the same degree sequence (and hence the same degree distribution) such that .
(a): For with at most vertices, if contains two adjacent vertices and with degrees more then then define and as follows: Let contain and a disjoint edge . Let contain with the edge removed and two non-adjacent vertices and adjacent to and respectively. These two graphs obviously have the same degree sequence and whereas .
Hence, the cases remaining to be considered are such where is a forest of stars. If there are at least two stars in that are -stars or larger then call these and . Let be and define as follows. Connect the hubs of and . In addition, take a leaf from each of and , connect them and remove the edge between them and their respective hubs. Now and have the same degree sequence but and .
If contains only one star with hub that is not an edge and some disjoint edges then take an edge in which is disjoint from . Define by adding a vertex to and connecting it to and to a leaf of . Define by taking , removing the edge and the edge, and adding the edge and the edge. Both graphs have the same degree sequence, but if is a -star or larger we have and ; if is a -star we have and .
Finally, the only case that is left is when is collection of three or more disjoint edges: , , , and a set of disjoint edges. Let be the union of the cycle , the edge, and ; let be the union of the path and . Both graphs have the same degree sequence but whereas .
(b): By (a) it is enough to work with with vertices that is not one of those mentioned. Collections of disjoint edges and stars (including forest of edges) have been covered in the proof of (a). If is disconnected and not one of these, then take two connected components with edges and in each component. Let be , and define by removing and and connecting to and to . We have that whereas .
Hence, henceforth, assume that is connected. If there is an induced or in , i.e., an induced path or cycle with nodes, then define and as follows: For , add a vertex and connect it to . For , add and connect it to , remove , and add . The graphs have the same degree sequence, but whereas .
If there are no induced or the graph is trivially perfect (see Brandstadt et al. (1999) Section 7.1.2) and there is a vertex that is adjacent to all other vertices in .
If among the neighbours of , there are two non-adjacent pairs of vertices and an edge we have the following two cases:
- Among the neighbours of , suppose that there are two non-adjacent pairs of vertices and and a pair of adjacent vertices , where may be the same as . In this case, define by adding and connecting it to , and define by adding and connecting it to , removing , and connecting . The graphs have the same degree sequence, but whereas .
2): Among the neighbours of , suppose that there are two non-adjacent pairs of vertices and and a pair of adjacent vertices , where may be the same as . In this case, define by adding and connecting it to , and define by adding and connecting it to , adding , and removing . The graphs have the same degree sequence, but whereas .
Therefore, the only cases that are left are stars, the complete graph, or the complete graph with one edge removed. This completes the proof.
∎
In general, if and have the same degree distributions, the fact that their probabilities should be identical induces a linear restriction on the parameters and hence the set of exchangeable and summarized distributions on a nodeset is again a linear exponential family with dimension equal to the number of distinct degree distributions for simple graphs on nodes; however, the restrictions induced on the parameters in (14) may be complicated, as illustrated in the example below.
Similarly, if and have the same degree distributions, a linear restriction is induced on the parameters in (13). This is also illustrated in the example below.
Example 8**.**
For networks with at most four nodes, the isomorphism class of a network is uniquely determined by the degree distribution, so we consider networks with five nodes. For most such networks, the isomorphism class is also identified by the degree distribution, the six exceptions being displayed in Fig. 6, where the node degrees of and both are , those of and , and those of and .
It thus follows that for any choice of canonical parameters , the distribution of an exchangeable network on five nodes is summarized if and only if , , and . Using the exponential representation (14) we obtain that the canonical parameters must satisfy three linear relations. From the relation we get
[TABLE]
where we have used Lemma 2 to infer that all terms of the form must cancel if is a -star or U=\hskip 3.50006pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}. Similarly, we get from that
[TABLE]
and a somewhat more involved linear relation appears from the fact that . Similarly using (13) we obtain that the Möbius parameters must satisfy three linear relations. The simplest of the three relations with Möbius parameters for is
[TABLE]
We refrain from giving details of the corresponding constraints with Möbius parameters obtained from and as well as with canonical parameters obtained from .
Corollary 1**.**
Let be a finite unlabelled graph without isolated nodes. If there is a function so that for all networks of any size, then , a -star, or U={\hskip 3.50006pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}}, the disjoint union of two independent edges.
Proof.
If this is true for all networks, we can always choose sufficiently large for Lemma 3, (a) to apply.
∎
5.2 The SE* model
Motivated by the results from the previous section, we can now simplify the exchangeable ERGM (14) and define an ERGM by the expression
[TABLE]
where is the number of -stars in , , and each of the parameters \nu_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}},\nu_{1},\ldots,\nu_{|\mathcal{N}|-1} vary independently on the real line.
We shall refer to this ERGM as the SE-model*, reflecting that all distributions within the model are both summarized and exchangeable, the star indicating that the model is not containing all such distributions. This is because neither the number of complete sub-graphs on nor the number of complete sub-graphs on nodes with one edge removed are sufficient statistics for the SE* model, so that, in virtue of Lemma 2 and Lemma 3, there exist summarized exchangeable network distributions that are not in the SE* model. The distributions implied by the marginal beta model, discussed below, are such an example. We reserve the term SE-model for the set of all distributions that are exchangeable and summarized and DE-model for the set of all distributions that are exchangeable and dissociated.
We note the similarities between the SE*-model (27) and the exchangeable Markov model (EM-model) of Frank and Strauss (1986), exponential family of the form
[TABLE]
which differs from SE*-model only by \sigma_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-4.pdf}}}}}\nu_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-4.pdf}}}}} replacing \sigma_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}}\,\nu_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-3.pdf}}}}}. Though nearly identical, the justifications behind the EM and the SE* models are quite different. The EM model results from postulating certain symmetric Markov properties for the edges, covered by case (b) in 3. On the other hand, the SE* model arises as a convenient sub-model of the class of summarized and exchangeable network distributions. The intersection of the SE*-model and the EM-model only contains the counts for -stars in the exponent, and so does the intersection of the SE-model and the EM model. The resulting SEM-model is the ERGM with sufficient statistics given by the entries degree distribution, studied in Sadeghi and Rinaldo (2014). This is because there is a one-to-one linear transformation between the number of -stars of and the degree distribution of , as demonstrated in the proof of Lemma 2.
From the computational standpoint, parameter estimation in the SE* model and in the EM model can be carried out by the same procedures, such as pseudo-maximum likelihood estimation and MCMCMLE methods – see, e.g., Strauss and Ikeda (1990), Geyer and Thompson (1992) and Hunter et al. (2008) – and is likely to be just as problematic.
5.3 The marginal beta model
Here we consider the marginal beta model, an example of a family of network distributions that are dissociated, exchangeable, and summarized, and consequently satisfy all the conditions discussed above.
Consider the beta model defined in (7). Suppose that are independent and identically distributed real-valued random variables with distribution . Then the expression (7) can be understood as the conditional distribution of given and and thus the density satisfies the factorization for the directed acyclic graph (DAG) that consists of and as vertices and arrows from each to all the , .
Figure 7 illustrates the case for .
Indeed the exact same graph also illustrates the Markov structure for a general -matrix, where just should be replaced by uniformly distributed and the arrows represent the s.
By marginalization over and conditioning on the variables in the DAG model the resulting dependence structure can in general be captured by mixed graphs; for further details see, for example, Richardson and Spirtes (2002) and Sadeghi (2013). For example, by conditioning on the we obtain the empty graph with all the as vertices. This implies the complete independence of , corresponding to the dyadic independence in the beta model (7). By marginalizing over , the bidirected incidence graph appears, as studied extensively in this paper and exemplified in Figure 4. As discussed before, this implies that after marginalization over , the network is dissociated. We refer to this model as the marginal beta model; the dissociatedness also follows immediately from the following “de Finetti type” representation for the distribution of :
[TABLE]
where denotes the -fold product distribution from . The function in Proposition 1 is in fact as described in (7). It clearly follows from (28) that the marginal beta distribution is exchangeable; in addition, (8) implies that the marginal beta model is summarized.
6 Consistent systems of network models
In this section we shall consider systems of network models and their relation to each other. We let be the basic set of network nodes and consider models for finite subsets . The basic set may be finite or countably infinite. If is a distribution of a random network with nodeset and we let denote the distribution of the induced subnetwork under . Thus the marginalization operator takes as input a probability distribution on and returns the induced or marginal probability distribution over corresponding to the dyads with endpoints in . By constructions, marginalization operators satisfy the property
[TABLE]
for any , so they form a projective system. A collection of random network models for all finite subsets of is said to be (strongly) consistent if for every in the collection it holds that
[TABLE]
in other words it holds that the marginal to a subnetwork of any network model is identical to the corresponding network model for the subnetwork. Thus a strongly consistent system forms a projective statistical field in the sense of Lauritzen (1988). We say the system is weakly consistent if
[TABLE]
i.e. marginalization of any network model for a larger nodeset is not in conflict with the network model for the subnetwork.
We first note that the bidirected Markov property itself is always marginalizable in the sense that if is bidirected Markov w.r.t. a graph , then the marginal distribution of is automatically Markov w.r.t. the induced subgraph , whereas the similar statement is generally not true for the undirected Markov property.
This holds in particular for the random network models when is the bidirected incidence graph. In other words, the system of dissociated random network models is weakly consistent. The same holds for the system of exchangeable random networks , and when both exchangeability and dissociatedness are combined to form the system of exchangeable and dissociated random networks , or when considering the system of models , which are all Markov w.r.t. the bidirected complements of the line graphs.
Correspondingly, the Möbius parametrization is marginalizable in the sense that if is a random network with nodeset and , the Möbius parameters for the induced subnetwork are identical to the corresponding parameters for :
[TABLE]
From any weakly consistent system we can construct a strongly consistent system , by letting
[TABLE]
provided that the intersection is non-empty. Thus each family in the system consists exactly of those distributions that can be obtained as marginals of distributions from models on arbitrarily large nodesets. We shall thus say that the elements in the strongly consistent variant of a weakly consistent system are completely extendable.
We note that, for example, the system of exchangeable random networks is not strongly consistent. In Example 2, the estimated distributions are easily seen not to be extendable to a network with more than four nodes. In general it is a difficult issue to determine whether a given system of Möbius parameters is extendable to a larger network with nodeset , i.e. if there exists a system of Möbius parameters , such that (29) holds.
Nevertheless, we can characterize the strongly consistent sequences of exchangeable random networks as follows. Below, if is infinite, we let be the set of all finite subsets of , which, as before, we represent as finite edge-induced subgraphs of the complete graph on .
Proposition 6**.**
All strongly consistent and infinitely extendable systems of exchangeable Möbius parameters are those which satisfy the conditions
- (a)
; 2. (b)
* if and are in ;* 3. (c)
for all non-negative integers and all with
[TABLE]
where we recall that is the number of graphs in that contain as a subgraph. In addition, the sequences of Möbius parameters corresponding to the extremal distribution are those satisfying the dissociated property that if has connected components then .
Proof.
From (29) it follows that any system of valid Möbius parameters is consistent and as previously argued, these are valid if and only if they satisfy and the non-negativity restriction in equation (13) above. We have also previously argued that the additional product condition characterizes dissociated networks which are the extreme points according to de Finetti’s theorem (Proposition 1).
∎
It is worth pointing out that, when , if is a random network on whose distribution is extremal (and therefore dissociated), then, for each non-empty with ,
[TABLE]
where denotes the set of edges in and is some measurable, symmetric function from into .
Using the arguments in Chapter 14 of Aldous (1985) (see also Kallenberg, 2005, for a generalization), de Finetti’s theorem (Proposition 1) and Proposition 6 yields the following result.
Corollary 2**.**
The convex set of feasible Möbius parameters are those which can be represented as
[TABLE]
where and is a (not necessarily unique) measurable function from to . The extreme points of this convex set correspond to the cases in which the function is a constant function of , for almost all .
Still, these characterizations are quite implicit. It seems difficult to obtain more explicit characterizations. For example, Figure 16.1 on page 288 of Lovász (2012) displays the area of variation of (z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-1.pdf}}}}},z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-4.pdf}}}}}) as given in (30) for all exchangeable and dissociated infnite networks, and this is a very complicated region. The set of infinitely extendable pairs is given by the convex hull of the region (Bollobás, 1976).
Using Corollary 2 we can show that strongly consistent systems of exchangeable models which are Markov w.r.t. bidirected complement of the incidence graph must be Erdös–Renyi models. First, we need a preliminary lemma. The lemma and proof are analogues of Theorem 10 in Diaconis and Freedman (1981).
Lemma 4**.**
Assume is an exchangeable random network with an infinitely extendable set of Möbius parameters satisfying, for some ,
[TABLE]
Then, the edges in are independent and identically distributed with , i.e. is Erdös–Renyi.
Proof.
Since all elements are exchangeable, we have from (31) that
[TABLE]
But also
[TABLE]
Thus,
[TABLE]
and hence
[TABLE]
which in turn implies that, if we assume ,
[TABLE]
Next, we have that
[TABLE]
and hence we conclude as above that, for almost all ,
[TABLE]
We would now wish to let in the above equation but this might just be the exceptional set. However, we may extend periodically to be defined on by letting
[TABLE]
so that (32) still holds almost surely. Then, using Lebesgue’s theorem, we can write
[TABLE]
As above, we conclude, for almost all , that and hence we have , implying that all edges are independent. Hence, the model is Erdös–Renyi.
∎
Proposition 7**.**
Consider a countable nodeset and a strongly consistent family of random network models; if all elements of are exchangeable and Markov w.r.t. the complement of the bidirected line graph , then they are all Erdös–Renyi models.
Proof.
We assume that . By exchangeability, we have that, for some , z_{\hskip 2.45004pt{\vbox{\hbox{\includegraphics[scale={0.05}]{figureex2-1.pdf}}}}}=\mathbb{P}(X_{12}=1)=\eta. Since X_{12}\mbox{\,\perp!!!\perp\,}X_{23}, we also have that
[TABLE]
where the last equality is obtained as in the proof of Lemma 4. The bidirected complement of the incidence graph for contains the edges
[TABLE]
Since all the distributions are Markov w.r.t. this graph we have
[TABLE]
and the conclusion now follows from Lemma 4.
∎
Analogous results hold true if the Markov assumption in Proposition 7 is replaced with Markov properties with respect to the undirected incidence graph as in the Frank–Strauss models or the undirected complement of the line graph, as in the Kneser models. The proof of these facts are quite elementary and we abstain from giving the details. In fact these sequences are not even weakly consistent, as marginals of undirected Markov distributions are not Markov with respect to the corresponding induced subgraph.
Finally we hasten to point out that indeed the system of marginal beta models is strongly consistent by construction and that this clearly remains true if the mixing distribution is assumed to belong to any specific subset of distributions .
7 Discussion
We have derived a complete characterization of possible Markov properties of exchangeable network models, and studied some of the implied models. We also consider summarized exchangeable network models in which the probability of a network is only a function of its degree distributions, and relate them to the other models we discuss in the paper. Overall, our findings have unveiled several interesting properties of exchangeable network models and established connections among seemingly disparate concepts used in network analysis, the probability literature on exchangeable arrays and the discrete mathematics literature on graph limits.
While we have focused on exchangeable and extendable network models, we point out that these are by no means the only interesting network models. In fact, many other parametric models, not fulfilling those properties, can also be successfully deployed to represent and study networks. Examples include the beta model (Chatterjee et al., 2011; Yan and Xu, 2013; Rinaldo et al., 2013) and the sparse Bernoulli model considered in Krivitsky and Kolaczyk (2015); see also the remarks in the introduction on partial conditional independence and on weakened forms of exchangeability.
We conclude with a final remark on the computational difficulties associated with fitting the various models discussed in the paper. As it is very often the case with ERGMs, model fitting can be rather challenging, and our models are no exception. For the DE and DSE models, one could in principle rely on the algorithms proposed in Drton and Richardson (2008), appropriately modified to allow for zero counts and for the fact that the MLE of the model parameters is nearly all cases on the boundary of the parameter space. However such algorithms may not scale well with the network size. As for the model given in (14), and its submodels, the SE and SE* and the EM models, parameter estimation can be carried out using pseudo-maximum likelihood and MCMC-based methods, (see, e.g. Strauss and Ikeda, 1990; Geyer and Thompson, 1992), implemented, for instance, in the R package ergm (Hunter et al., 2008). In all these cases, however, we expect that fitting could be problematic, as these procedures do not scale well with the network size and MCMC convergence can be very slow or hard to assess. In particular, we expect that the issues of degeneracy (see Schweinberger, 2011; Rinaldo et al., 2009) plaguing many ERGMs will also impact estimation of these models.
Acknowledgements
The authors are grateful to Éva Czabarka for proving an earlier version of Lemma 3. Alessandro Rinaldo and Kayvan Sadeghi were partially supported by AFOSR grant FA9550-14-1-0141.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Aldous (1981) Aldous, D. (1981). Representations for partially exchangeable random variables. Journal of Multivariate Analysis 11 , 581–598.
- 2Aldous (1985) Aldous, D. (1985). Exchangeability and related topics. In P. Hennequin (Ed.), École d’Été de Probabilités de Saint–Flour XIII — 1983 , pp. 1–198. Heidelberg: Springer-Verlag. Lecture Notes in Mathematics 1117.
- 3Baddeley and Møller (1989) Baddeley, A. and J. Møller (1989). Nearest-neighbour Markov point processes and random sets. International Statistical Review 57 , 89–121.
- 4Barndorff-Nielsen (1978) Barndorff-Nielsen, O. E. (1978). Information and Exponential Families in Statistical Theory . Chichester, UK: John Wiley and Sons.
- 5Bollobás (1976) Bollobás, B. (1976). Relations between sets of complete subgraphs. In C. S. J. A. Nash-Williams and J. Sheehan (Eds.), Proceedings of the 5th British Combinatorics Conference .
- 6Boutilier et al. (1996) Boutilier, C., N. Friedman, M. Goldszmidt, and D. Koller (1996). Context-specific independence in Bayesian networks. In Proceedings of the Twelfth International Conference on Uncertainty in Artificial Intelligence , UAI’96, San Francisco, CA, USA, pp. 115–123. Morgan Kaufmann Publishers Inc.
- 7Brandstadt et al. (1999) Brandstadt, A., V. B. Le, and J. Spinrad (1999). Graph Classes: A Survey . SIAM Monographs on Discrete Mathematics and Applications. Philadelphia: Society of Industrial and Applied Mathematics.
- 8Caron and Fox (2017) Caron, F. and E. Fox (2017). Sparse graphs using exchangeable random measures. Journal of Royal Statistical Society, Series B 79 , 1295–1366.