Network Comparison: Embeddings and Interiors

TL;DR

This paper introduces a method to compare networks by embedding them into a metric space, explores the concept of network interiors to improve approximation, and demonstrates practical discrimination of different network models.

Contribution

It defines a valid network distance metric based on optimal embeddings and introduces interior-based methods to enhance comparison accuracy.

Findings

Network distance is a valid metric in network space.

Interior comparison aligns with actual network distances.

Method effectively discriminates between different network models.

Abstract

This paper presents methods to compare networks where relationships between pairs of nodes in a given network are defined. We define such network distance by searching for the optimal method to embed one network into another network, prove that such distance is a valid metric in the space of networks modulo permutation isomorphisms, and examine its relationship with other network metrics. The network distance defined can be approximated via multi-dimensional scaling, however, the lack of structure in networks results in poor approximations. To alleviate such problem, we consider methods to define the interiors of networks. We show that comparing interiors induced from a pair of networks yields the same result as the actual network distance between the original networks. Practical implications are explored by showing the ability to discriminate networks generated by different models.

Equations

$$r_X(x,x^{\prime })=r_Y(\varphi (x),\varphi (x^{\prime })).$$

$$\displaystyle\Gamma_{X,Y}(C)\!:=\!\max_{(x,y),(x^{\prime},y^{\prime})\in C}\Big{|}r_{X}(x,x^{\prime})-r_{Y}(y,y^{\prime})\Big{|}.$$

$$\displaystyle d_{\text{C}}(N_{X},N_{Y}):=\min_{C\in{\mathcal{C}}(X,Y)}\Big{\{}\Gamma_{X,Y}(C)\Big{\}}.$$

$$\displaystyle\Delta_{X,Y}(\phi):=\max_{x,x^{\prime}\in X}\Big{|}r_{X}(x,x^{\prime})-r_{Y}(\phi(x),\phi(x^{\prime}))\Big{|}.$$

$$\displaystyle d_{\text{PE}}(N_{X},N_{Y}):=\min_{\phi:X\rightarrow Y}\Big{\{}\Delta_{X,Y}(\phi)\Big{\}}.$$

$$d_{\text{E}}(N_X,N_Y):=\max \left\{d_{\text{PE}}(N_X,N_Y),d_{\text{PE}}(N_Y,N_X)\right\}.$$

$$d_{\text{C}}(N_X,N_Y)=\underset{\begin{array}{c}\varphi :X\to Y\\ \psi :Y\to X\end{array}}{\min }\max \left\{{\Delta }_{X,Y}(\varphi ),{\Delta }_{Y,X}(\psi ),{\delta }_{X,Y}(\varphi ,\psi )\right\},$$

$${\delta }_{X,Y}(\varphi ,\psi )=\underset{x\in X,y\in Y}{\max }\left|r_X(x,\psi (y))-r_Y(\varphi (x),y)\right|.$$

$$d_{\text{E}}(N_X,N_Y)\le d_{\text{C}}(N_X,N_Y),$$

$$\min$$

$$s.t.$$

$$m_b=p_b+w_{ab}-w_{bc},$$

$$m_c=p_c+w_{ac}+w_{bc}.$$

$$\min$$

$$s.t.$$

$$m_b=p_b+w_{ab}-w_{bc},$$

$$m_c=p_c+w_{ac}+w_{bc}.$$

$$s_X(p,m)=\left|w_{ab}^{\star }\right|r_X(a,b)+\left|w_{bc}^{\star }\right|r_X(b,c)+\left|w_{ac}^{\star }\right|r_X(a,c).$$

$$\left\{w_{ij}^{\star }\right\}=\mathrm{argmin}$$

$$s.t.$$

$$s_X(p,m)=\sum _{i,j\in X,i<j}\left|w_{ij}^{\star }\right|r_X(i,j).$$

$$\displaystyle\Delta_{S_{X},S_{Y}}(\phi):=\max_{x,x^{\prime}\in S_{X}}\Big{|}s_{X}(x,x^{\prime})-s_{Y}(\phi(x),\phi(x^{\prime}))\Big{|}.$$

$$\displaystyle d_{\text{PE, S}}(N_{X},N_{Y}):=\min_{\phi:S_{X}\rightarrow S_{Y}\mid\phi(x)\in Y,\forall x\in X}\Big{\{}\Delta_{S_{X},S_{Y}}(\phi)\Big{\}}.$$

$$d_{\text{PE, S}}(N_X,N_Y)=d_{\text{PE}}(N_X,N_Y),\text{for all }{N}_X,N_Y.$$

$$[\varphi (x){]}_i=\sum _{j\in X}1\left\{\hat{\varphi }(j)=i\right\}{x}_j,$$

$$[\psi (y){]}_j=\sum _{i\in Y}1\left\{\hat{\psi }(i)=j\right\}{y}_i.$$

$$\sum _{i\in Y}[\varphi (x){]}_i=\sum _{i\in Y}\sum _{j\in X}1\left\{\hat{\varphi }(j)=i\right\}{x}_j=\sum _{j\in X}{x}_j=1,$$

$$\displaystyle\Delta_{Q_{X},Q_{Y}}(\phi):=\max_{x,x^{\prime}\in Q_{X}}\Big{|}{\hat{r}}_{X}(x,x^{\prime})-{\hat{r}}_{Y}(\phi(x),\phi(x^{\prime}))\Big{|}.$$

$$\displaystyle d_{\text{PE, Q}}(N_{X},N_{Y})\!:=\!\min_{\phi:Q_{X}\rightarrow Q_{Y}\mid\phi(x)\in Y,\forall x\in X}\Big{\{}\Delta_{Q_{X},Q_{Y}}(\phi)\Big{\}}.$$

$$d_{\text{PE, Q}}(N_X,N_Y)=d_{\text{PE}}(N_X,N_Y),$$

$$d_{\text{PE}}(N_X,N_Y)\le {\Delta }_{X,Y}(\varphi )=0.$$

$$d_{\text{PE}}(N_X,N_Z)={\Delta }_{X,Z}(\varphi ),d_{\text{PE}}(N_Z,N_Y)={\Delta }_{Z,Y}(\psi ).$$

$$d_{\text{PE}}(N_X,N_Y)$$

$${\Delta }_{X,Y}$$

Full text

I Introduction

The field of network science is predicated on the empirical observation that network structure carries important information about phenomena of interest. Network structures have been observed to be fundamental in social organizations [1] and differences in the structure of brain networks have been shown to have clinical value in neurology and psychology [2]. The fundamental value of network structure has led to an extensive literature on network identification that is mostly concerned with the identification of network features that serve as valuable discriminators in different contexts. Examples of these features and application domains are clustering coefficients [3] and motifs [4] in social networks; neighborhood topology [5], betweenness [6], and wavelets [7] in protein interaction networks; as well as graphlet degree distributions [8], graph theoretic measures [9], and single linkage dendrogram [10] and homology [11] in different contexts. However valuable, it is material to recognize that it is possible for very different networks to be indistinguishable from the perspective of specific features, or, conversely, to have similar networks that differ substantially on the values of some features. One way to sidestep this limitation it to define and evaluate proper network distances. This is the objective of this paper.

Without getting into the details of how a valid distance might be defined, it is apparent that their computation is bound to be combinatorial. Indeed, since permutations of unlabeled nodes result in identical networks, distances must rely on comparison between a combinatorial number of node correspondences – evaluating distances is relatively simpler if nodes are labeled [12, 13, 14]. An important observation in this regard is that the space of finite metric spaces is a subset of the space of networks composed of those whose edges satisfy the triangle inequality. This observation is pertinent because there is a rich literature on the comparison of metric spaces that we can adopt as a basis for generalizations that apply to the comparison of networks. Of particular interest here are the Gromov-Hausdorff distance, which measures the size of the smallest modification that allows the spaces to be mapped onto each other [15, 16], and the partial embedding distance, which measures the size of the smallest modification that allows a space to be embedded in the other [17, 18, 19]. The computation of either of these distances is intractable. However, Gromov-Hausdorff distances can be tractably approximated using homological features [20] and embedding distances can be approximated using multi-dimensional scaling (MDS) [17].

In prior work we have defined generalizations of the Gromov-Hausdorff distance [21, 22] to networks, and utilized homological features for computationally tractable approximate evaluations [23, 24]. Our starting point here is the partial embedding distance for metric spaces [17, 18, 19, 16]. Our goal is to generalize embedding distances to arbitrary networks and utilize MDS techniques for their approximate computation.

I-A Organization And Contributions

Networks, metrics, embedding metrics, and the Gromov-Hausdorff distance for networks are defined in Section II. The issue of defining an embedding distance for networks is addressed in Section III. The idea of an embedding distance $d(A,B)$ is to analyze how much we have to modify network $A$ to make it a subset of network $B$. This is an asymmetric relationship. In particular, having $d(A,B)=0$ means that network $A$ can be embedded in network $B$ but the opposite need not be true. The first contribution of this paper is to:

• (i)

Define network embeddings and a corresponding notion of partial embedding distances. Partial embedding distances define an embedding metric $d$ such that $d(A,B)=0$ if and only if $A$ can be embedded in $B$.

We attempt to use the MDS techniques in [18] to approximate the computation of embedding distances but observe that the methodology yields poor results – see Figure 1 for an illustration of why this is not unexpected. To improve these results we observe that when edge dissimilarities satisfy a triangle inequality, an Euclidean interior is implicitly defined. In the case of arbitrary networks this is not true and motivates the definition of the interior of a network that we undertake in Section IV. The second contribution of this paper is to:

• (ii)

Provide a definition of the interior of a network. The interior of a set of nodes is the set of points that can be written as convex combinations of the nodes. When the network forms a metric space, the dissimilarity between a pair of points in the interior is the distance on the shortest path between the pair. When the dissimilarities in the network do not form a metric space, e.g. representing travel time between nodes, such construction would yield conflict. The problem can be solved by defining the dissimilarity between a pair of points as the travel time on the shortest path between the pair.

Having the ability to extend networks into their interiors, we extend different networks and compute partial embedding distances between their extensions. In principle, distances between two networks and their respective extensions need not be related. In Section IV-A we show that a restriction in the embedding of the extended networks renders them identical. Our third and most important contribution is to:

• (iii)

Define embeddings for extended networks such that points in one of the original networks – prior to extension – can only be embedded into original points of the other network. We show that the embedding distance that results from this restriction is the same embedding distance between the original networks.

The definition of interior is somewhat arbitrary, however because of (iii), the practical implication of interior definition is justified. We point out that a network extension is a dense set that includes all the convex combinations of sets of points. To make interior extensions practical we consider samplings of the interior in Section IV-B. It is not difficult to show in light of Contribution (iii) that the embedding distance between a a pair of networks extended to samples of their interiors is also identical to the embedding distance between the original pair of networks – if the restriction in the mapping of original nodes is retained.

We exploit Contributions (ii) and (iii) to approximate the computation of embedding distances using the MDS techniques in [18] but applied to networks extended to their interiors. The definition of an interior markedly improves the quality of MDS distance approximations. We illustrate this fact in Section V with an artificial illustrative example and also demonstrate the ability to discriminate networks with different generative models. We only extend by adding points that are mid-points of original nodes in the networks, in order to make the process computationally tractable. The small number of points considered in interiors is sufficient to distinguish networks of different processes, despite that the original networks may possess different number of nodes.

II Preliminaries

A network is defined as a pair $N_{X}=(X,r_{X})$, where $X$ is a finite set of nodes and $r_{X}:X^{2}=X\times X\rightarrow{\mathbb{R}}_{+}$ is a function encoding dissimilarity between pairs. For $x,x^{\prime}\in X$, values of this function are denoted as $r_{X}(x,x^{\prime})$. We assume that $r_{X}(x,x^{\prime})=0$ if and only if $x=x^{\prime}$ and we further restrict attention to symmetric networks where $r_{X}(x,x^{\prime})=r_{X}(x^{\prime},x)$ for all pairs of nodes $x,x^{\prime}\in X$. The set of all such networks is denoted as ${\mathcal{N}}$.

When defining a distance between networks we need to take into consideration that permutations of nodes amount to relabelling nodes and should be considered as same entities. We therefore say that two networks $N_{X}=(X,r_{X})$ and $N_{Y}=(Y,r_{Y})$ are isomorphic whenever there exists a bijection $\phi:X\rightarrow Y$ such that for all points $x,x^{\prime}\in X$,

[TABLE]

Such a map is called an isometry. Since the map $\phi$ is bijective, (1) can only be satisfied when $X$ is a permutation of $Y$. When networks are isomorphic we write $N_{X}\cong N_{Y}$. The space of networks where isomorphic networks $N_{X}\cong N_{Y}$ are represented by the same element is termed the set of networks modulo isomorphism and denoted by ${\mathcal{N}}\mod\cong$. The space ${\mathcal{N}}\mod\cong$ can be endowed with a valid metric [21, 22]. The definition of this distance requires introducing the prerequisite notion of correspondence [25, Def. 7.3.17].

Definition 1

A correspondence between two sets $X$ and $Y$ is a subset $C\subseteq X\times Y$ such that $\forall~{}x\in X$, there exists $y\in Y$ such that $(x,y)\in C$ and $\forall~{}y\in Y$ there exists $x\in X$ such that $(x,y)\in C$. The set of all correspondences between $X$ and $Y$ is denoted as ${\mathcal{C}}(X,Y)$.

A correspondence in the sense of Definition 1 is a map between node sets $X$ and $Y$ so that every element of each set has at least one correspondent in the other set. Correspondences include permutations as particular cases but also allow mapping of a single point in $X$ to multiple correspondents in $Y$ or, vice versa. Most importantly, this allows definition of correspondences between networks with different numbers of elements. We can now define the distance between two networks by selecting the correspondence that makes them most similar as stated next.

Definition 2

Given two networks $N_{X}=(X,r_{X})$ and $N_{Y}=(Y,r_{Y})$ and a correspondence $C$ between the node spaces $X$ and $Y$ define the network difference with respect to $C$ as

[TABLE]

The network distance between $N_{X}$ and $N_{Y}$ is then defined as

[TABLE]

For a given correspondence $C\in{\mathcal{C}}(X,Y)$ the network difference $\Gamma_{X,Y}(C)$ selects the maximum distance difference $|r_{X}(x_{1},x_{2})-r_{Y}(y_{1},y_{2})|$ among all pairs of correspondents – we compare $r_{X}(x_{1},x_{2})$ with $r_{Y}(y_{1},y_{2})$ when the points $x_{1}$ and $y_{1}$, as well as the points $x_{2}$ and $y_{2}$, are correspondents. The distance in (3) is defined by selecting the correspondence that minimizes these maximal differences. The distance in Definition 2 is a proper metric in the space of networks modulo isomorphism. It is nonnegative, symmetric, satisfies the triangle inequality, and is null if and only if the networks are isomorphic [21, 22]. For future reference, the notion of metric is formally stated next.

Definition 3

Given a space ${\mathcal{S}}$ and an isomorphism $\cong$, a function $d:{\mathcal{S}}\times{\mathcal{S}}\rightarrow{\mathbb{R}}$ is a metric in ${\mathcal{S}}\mod\cong$ if for any $a,b,c\in{\mathcal{S}}$ the function $d$ satisfies:

• (i) Nonnegativity.

$d(a,b)\geq 0$.

• (ii) Symmetry.

$d(a,b)=d(b,a)$.

• (iii) Identity.

$d(a,b)=0$* if and only if $a\cong b$.*

• (iv) Triangle inequality.

$d(a,b)\leq d(a,c)+d(c,b)$.

A metric $d$ in ${\mathcal{S}}\mod\cong$ gives a proper notion of distance. Since zero distances imply elements being isomorphic, the distance between elements reflects how far they are from being isomorphic. The distance in Definition 2 is a metric in space ${\mathcal{N}}\mod\cong$. Observe that since correspondences may be between networks with different number of elements, Definition 2 defines a distance $d_{\text{C}}(N_{X},N_{Y})$ when the node cardinalities $|X|$ and $|Y|$ are different. In the particular case when the functions $r_{X}$ satisfy the triangle inequality, the set of networks ${\mathcal{N}}$ reduces to the set of metric spaces ${\mathcal{M}}$. In this case the metric in Definition 2 reduces to the Gromov-Hausdorff (GH) distance between metric spaces. The distances $d_{\text{C}}(N_{X},N_{Y})$ in (3) are valid metrics even if the triangle inequalities are violated by $r_{X}$ or $r_{Y}$ [21, 22].

A related notion is that of an isometric embedding. We say that a map $\phi:X\rightarrow Y$ is an isometric embedding from $N_{X}=(X,r_{X})$ to $N_{Y}=(Y,r_{Y})$ if (1) holds for all points $x,x^{\prime}\in X$. Since $r_{X}(x,x^{\prime})=r_{Y}(\phi(x),\phi(x^{\prime}))$ for any $x,x^{\prime}\in X$, $r_{X}(x,x^{\prime})>0$ for $x\neq x^{\prime}$ and $r_{Y}(y,y)=0$, the map $\phi$ is injective. This implies that the condition can only be satisfied when $N_{X}=(X,r_{X})$ is a sub-network of $N_{Y}=(Y,r_{Y})$. Such a map is called an isometric embedding. When $N_{X}$ can be isometrically embedded into $N_{Y}$, we write $N_{X}\sqsubseteq N_{Y}$. Related to the notion of isometric embedding is the notion of an embedding metric that we state next.

Definition 4

Given a space ${\mathcal{S}}$ and an isometric embedding $\sqsubseteq$, a function $d:{\mathcal{S}}\times{\mathcal{S}}\rightarrow{\mathbb{R}}$ is an embedding metric in ${\mathcal{S}}$ if for any $a,b,c\in{\mathcal{S}}$ the function $d$ satisfies:

• (i) Nonnegativity.

$d(a,b)\geq 0$.

• (ii) Embedding identity.

$d(a,b)=0$* if and only if $a\sqsubseteq b$.*

• (iii) Triangle inequality.

$d(a,b)\leq d(a,c)+d(c,b)$.

It is apparent that metrics are embedding metrics because bijections are injective, and that in general embedding metrics are not metrics because they are asymmetric. In this paper, we consider defining an embedding distance between networks and evaluate its relationship with the Gromov-Hausdorff distance (Section III). We then consider the problem of augmenting the networks by adding points to fill their “interior”. The interior is defined so that embedding metrics between the original networks and embedding metrics between these augmented spaces coincide (Section IV).

III Embeddings

As is the case with correspondences, mappings also allow definition of associations between networks with different numbers of elements. We use this to define the distance from one network to another network by selecting the mapping that makes them most similar as we formally define next.

Definition 5

Given two networks $N_{X}=(X,r_{X})$, $N_{Y}=(Y,r_{Y})$, and a map $\phi:X\rightarrow Y$ from node space $X$ to the node space $Y$, define the network difference with respect to $\phi$ as

[TABLE]

The partial embedding distance from $N_{X}$ to $N_{Y}$ is defined as

[TABLE]

Both, Definition 2 and Definition 5 consider a mapping between the node space $X$ and the node space $Y$, compare dissimilarities, and set the network distance to the comparison that yields the smallest value in terms of maximum differences. The distinction between them is that in (2) we consider correspondence, which requires each point in any node spaces ($X$ or $Y$) to have a correspondent in the other node space, whereas in (4) we examine mappings, which only require all points in node space $X$ to have one correspondent in the node set $Y$. Moreover, in (2), a node $x\in X$ may have multiple correspondents, however, in (4), a node $x\in X$ can only have exactly one correspondent. Except for this distinction, Definition 2 and Definition 5 are analogous since $\Delta_{X,Y}(\phi)$ selects the difference $|r_{X}(x_{1},x_{2})-r_{Y}(y_{1},y_{2})|$ among all pairs. The distance $d_{\text{PE}}(N_{X},N_{Y})$ is defined by selecting the mapping that minimizes these maximal differences. We show in the following proposition that the function $d_{\text{PE}}:{\mathcal{N}}\times{\mathcal{N}}\rightarrow{\mathbb{R}}_{+}$ is, indeed, an embedding metric in the space of networks.

Proposition 1

The function $d_{\text{PE}}:{\mathcal{N}}\times{\mathcal{N}}\rightarrow{\mathbb{R}}_{+}$ defined in (5) is an embedding metric in the space ${\mathcal{N}}$.

**Proof : **See Appendix A for proofs in Section III. ∎

The embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ from one network $N_{X}$ to another network $N_{Y}$ is not a metric due to its asymmetry. We can construct a symmetric version from $d_{\text{PE}}(N_{X},N_{Y})$ by taking the maximum from the embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ and $d_{\text{PE}}(N_{Y},N_{X})$. This would give us a valid metric distance in ${\mathcal{N}}\mod\cong$. A formal definition and theorem are shown next.

Definition 6

Given two networks $N_{X}=(X,r_{X})$, $N_{Y}=(Y,r_{Y})$, define the embedding distance between the pair as

[TABLE]

where partial embedding distances $d_{\text{PE}}(N_{X},N_{Y})$ and $d_{\text{PE}}(N_{X},N_{Y})$ are defined in Definition 5.

Theorem 1

The function $d_{\text{E}}:{\mathcal{N}}\times{\mathcal{N}}\rightarrow{\mathbb{R}}_{+}$ defined in (6) is a metric in the space ${\mathcal{N}}\mod\cong$.

Since embedding distances between two networks generate a well-defined metric, they provide a means to compare networks of arbitrary sizes. In comparing the embedding distance in (6) with the network distance in (3) we see that both find the bottleneck that prevents the networks to be matched to each other. It is not there surprising to learn that they satisfy the relationship that we state in the following proposition.

Proposition 2

The network distance $d_{\text{C}}(N_{X},N_{Y})$ defined in (3) can also be written as

[TABLE]

where the network differences $\Delta_{X,Y}(\phi)$ and $\Delta_{Y,X}(\psi)$ with respect to mappings $\phi$ and $\psi$ are defined in (4) and $\delta_{X,Y}(\phi,\psi)$ measures how far the mappings $\phi$ and $\psi$ are from being the inverse of each other, and is defined as

[TABLE]

A direct consequence of Lemma 2 is that the embedding distance (6) is a lower bound of the network distance (3).

Corollary 1

Function $d_{\text{E}}$ is a lower bound with $d_{\text{C}}$ in (3), i.e.

[TABLE]

for any networks $N_{X}$ and $N_{Y}$.

The relationships in Lemma 2 and Corollary 1 are extensions of similar analyses that hold for the Gromov-Hausdorff distance between metrics spaces, [26, 27]. As in the case of metric spaces, these results imply that the embedding distance $d_{\text{E}}(N_{X},N_{Y})$ can be used to lower bound the network distance $d_{\text{C}}(N_{X},N_{Y})$ [cf. (9)]. This value is in addition to the ability of the partial embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ of Definition 5 to measure how far the network $X$ is to being a subnetwork of network $Y$.

In the comparison of surfaces and shapes, the partial embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ has the attractive property of being approximable using multidimensional scaling techniques [17, 19]. Our empirical analysis shows that the use of analogous techniques to estimate $d_{\text{PE}}(N_{X},N_{Y})$ for arbitrary networks yields poor results and that this is related to how far the dissimilarities in $N_{X}$ and $N_{Y}$ are from satisfying the triangle inequality – see the example in Figure 1 and the numerical analysis in Section V. To improve the accuracy of multidimensional scaling estimates we propose to define the interior of a network by defining a space where dissimilarities between any pair of points represented by a convex combination of nodes in the given networks are defined (Section IV). We will further demonstrate that the proposed definition of the interior of a network is such that the partial embedding distances between networks with interiors are the same as the partial embedding distances between the corresponding original networks (Theorems 2 and 3). Empirical demonstrations will show that the comparison of networks with interiors using MDS techniques yields better results that are comparable to those obtained when comparing shapes and surfaces (Section V).

IV Interiors

We provide a different perspective to think of networks as semimetric spaces where: (i) There are interior points defined by convex combinations of given nodes. (ii) Dissimilarities between these interior points are determined by the dissimilarities between the original points. To substantiate the formal definition below (Definition 7) we discuss the problem of defining the interior of a network with three points. Such network is illustrated in Figure 2 where nodes are denoted as $a$, $b$, and $c$ and dissimilarities are denoted as $r_{X}$. Our aim is to induce a space $(S_{X},s_{X})$ where the dissimilarities in the induced space are $s_{X}:S_{X}\times S_{X}\rightarrow{\mathbb{R}}_{+}$. We require that $S_{X}$ preserve the distance of original points in $N_{X}$ such that $s_{X}(a,b)=r_{X}(a,b)$, $s_{X}(a,c)=r_{X}(a,c)$, and $s_{X}(b,c)=r_{X}(b,c)$.

Points inside the network are represented in terms of convex combinations of the original points $a$, $b$, and $c$. Specifically, a point $m$ in the interior of the network is represented by the tuple $(m_{a},m_{b},m_{c})$ which we interpret as indicating that $m$ contains an $m_{a}$ proportion of $a$, an $m_{b}$ proportion of $b$, and an $m_{c}$ proportion of $c$. Points $e$, $f$, and $g$ on Figure 2 contain null proportions of some nodes and are interpreted as lying on the edges. Do notice that although we are thinking of $m$ as a point inside the triangle, a geometric representation does not hold.

First we consider the case that the triangle inequality is satisfied by $r_{X}$. To evaluate the dissimilarities between $p$ represented by the tuple $(p_{a},p_{b},p_{c})$ and $m$ represented by the tuple $(m_{a},m_{b},m_{c})$ using dissimilarities in the original network, we need to find a path consisting of vectors parallel to the edges in the network that go to $m$ from $p$. Specifically, denote $w_{ab}$ as the proportion transversed in the direction from $a$ to $b$ in the path. For a positive value $w_{ab}$, compared to $p$, $m$ becomes more similar to $b$ by $w_{ab}$ units and less similar to $a$ by $-w_{ab}$ units; for a negative $w_{ab}$, compared to $p$, $m$ becomes more similar to $a$ and less similar to $b$. Proportion transversed in other directions, e.g. from $a$ to $c$ and from $b$ to $c$, are denoted as $w_{ac}$ and $w_{bc}$, respectively. For the path transversing $w_{ab}$ from $a$ to $b$, $w_{ac}$ from $a$ to $c$, and $w_{bc}$ from $b$ to $c$, the dissimilarity can be denoted as $|w_{ab}|r_{X}(a,b)+|w_{ac}|r_{X}(a,c)+|w_{bc}|r_{X}(b,c)$. There may be many different paths from $p$ to $m$, as illustrated in Figure 3. Out of all paths, only the one yielding the smallest distance should be considered. This means the dissimilarity $s_{X}(p,m)$ between $p$ and $m$ can be defined by solving the following problem,

[TABLE]

The constraints make sure that the path starts with tuple $(m_{a},m_{b},m_{c})$ and ends with tuple $(p_{a},p_{b},p_{c})$. This is like the definition of Manhattan distance. In fact, if Manhattan was a triangle with three endpoints and the roads in Manhattan were in a triangle grid, then the distance between any pair of points in Manhattan would be evaluated as in (10).

When relationships in $r_{X}$ do not satisfy triangle inequality, e.g. $r_{X}(a,b)+r_{X}(b,c)<r_{X}(a,c)$, however, the construction in (10) is problematic since the optimal solution in (10) would yield $s_{X}(a,c)=r_{X}(a,b)+r_{X}(b,c)$, which violates our requirement that $s_{X}(a,c)$ should be the same as $r_{X}(a,c)$. The problem arises because each segment in a given path contains two pieces of information – the proportion of transformation, and the dissimilarity created of such transformation. E.g. for the path segment $pq_{1}$ in Figure 3, it represents $w_{ab}^{\star}$ units of transformation from $a$ to $b$, and also denotes a dissimilarity between $p$ and $q_{1}$ as $|w_{ab}^{\star}|r_{X}(a,b)$. The two pieces of information unite when $r_{X}$ is a metric, however, create conflicts for dissimilarities in a general network. To resolve such issue, we could separate the amount of transformation from the dissimilarity incurred due to transformation. Firstly, we find the path with the smallest amount of transformation

[TABLE]

Then, for the optimal path $w_{ab}^{\star}$, $w_{ac}^{\star}$, and $w_{bc}^{\star}$ in (11), define the dissimilarity as the distance transversed on the path, i.e.

[TABLE]

The problem in (11) can always be solved since it is underdetermined due to the facts that $m_{a}+m_{b}+m_{c}=p_{a}+p_{b}+p_{c}=1$. It traces back to (10) when relationships in network are metrics. Moreover, it satisfy our requirement $s_{X}(a,b)=r_{X}(a,b)$, $s_{X}(a,c)=r_{X}(a,c)$, and $s_{X}(b,c)=r_{X}(b,c)$ for any networks. Regarding our previous example of a triangle-shaped Manhattan with three endpoints, suppose relationships in the network denote the amount of travel time between the endpoints. These relationship may not necessarily satisfy triangle inequalities. Suppose roads in Manhattan form a triangle grid, the problem in (11) is finding the shortest path between a pair of locations in Manhattan. The dissimilarity in (12) describes the travel time between this pair of locations using the shortest path.

Given any network with arbitrary number of nodes, we define the induced space as a generalization to the case for nodes with three nodes we developed previously.

Definition 7

Given a network $N_{X}=(X,r_{X})$ with $X=\{1,2,\dots,n\}$, the induced space $(S_{X},s_{X})$ is defined such that the space $S_{X}$ is the convex hull of $X$ with $S_{X}=\{m=(m_{1},m_{2},\dots,m_{n})\mid m_{i}\geq 0,\sum_{i\in X}m_{i}=1\}$. Given a pair of nodes $m,p\in S_{X}$, the path yielding the smallest amount of transformation from $p$ to $m$ is obtained through the problem

[TABLE]

The distance between $p$ and $m$ is then the distance traversed proportional to the original relationships weighted by the path,

[TABLE]

The induced space $S_{X}$ is the convex hull constructed by all nodes $i\in X$. Each node in the induced space $m\in S_{X}$ can be represented as a tuple $(m_{1},m_{2},\dots,m_{n})$ with $\sum_{i\in X}m_{i}=1$ where $m_{i}$ represents the percentage of $m$ inheriting the property of node $i\in X$. To come up with distance between pairs of points $p,m\in S_{X}$ with the respective tuple representation $(p_{1},p_{2},\dots,p_{n})$ and $(m_{1},m_{2},\dots,m_{n})$, we consider each edge in the original space $X$, e.g. from $i$ to $j$, represents one unit of cost to transform $i$ into $j$. All edges are considered similarly with one unit of cost to transform the starting node into the ending node. We want to find the smallest amount of cost to transform $p$ into $m$. This is solved via (13), which is always solvable since the problem is underdetermined due to the facts that $\sum_{i\in X}m_{i}=\sum_{i\in X}p_{i}=1$. This gives us the optimal path with weights $\{w_{ij}^{\star}\}$ meaning that the most cost-saving transformation from $p$ into $m$ is to undertaking $w_{ij}^{\star}$ unit of transformation along the direction of transforming $i$ into $j$. The distance in the induced space $s_{X}(p,m)$ is then the distance traversed proportional to the original relationships weighted by the path defined in (14).

Proposition 3

The space $(S_{X},s_{X})$ induced from $N_{X}=(X,r_{X})$ defined in Definition 7 is a semimetric space in $S_{X}$. Moreover, the induced space preserves relationships: when $p,m\in X$, $s_{X}(p,m)=r_{X}(p,m)$.

**Proof : **See Appendix B for proofs in Section IV. ∎

The semimetric established in Proposition 3 guarantees that the points in the induced space with their dissimilarity $s_{X}(p,m)$ are well-behaved. We note that semimetric is the best property we can expect, since the triangle inequality may not be satisfied even for the dissimilarities in the original networks. Next we show that the embedding distance is preserved when interiors are considered.

IV-A Distances Between Networks Extended To Their Interiors

Since semimetrics are induced purely from the relationships in the original network, a pair of networks $N_{X}$ and $N_{Y}$ can be compared by considering their induced space, as we state next.

Definition 8

Given two networks $N_{X}=(X,r_{X})$ and $N_{Y}=(Y,r_{Y})$ with their respective induced space $(S_{X},s_{X})$ and $(S_{Y},s_{Y})$, for a map $\phi:S_{X}\rightarrow S_{Y}$ from the induced space $S_{X}$ to the induced space $S_{Y}$ such that $\phi(x)\in Y$ for any $x\in X$, define the network difference with respect to $\phi$ as

[TABLE]

The partial embedding distance from $N_{X}$ to $N_{Y}$ measured with respect to the induced spaces is then defined as

[TABLE]

The partial embedding distance $d_{\text{PE, S}}(N_{X},N_{Y})$ with respect to the induced space in (16) is defined similarly as the partial embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ in (5) however considers the mapping between all elements in the induced spaces. Observe that we further require that the embedding satisfy $\phi(x)\in Y$ for any $x\in X$. This ensures the original nodes of network $X$ are mapped to original nodes of network $Y$. The restriction is incorporated because it makes the embedding distance $d_{\text{PE, S}}(N_{X},N_{Y})$ with respect to the induced spaces identical to the original embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ as we state next.

Theorem 2

The function $d_{\text{PE, S}}:{\mathcal{N}}\times{\mathcal{N}}\rightarrow{\mathbb{R}}_{+}$ defined in (16) is an embedding metric in the space ${\mathcal{N}}$ and yields the same distance as the function $d_{\text{PE}}$ defined in (5),

[TABLE]

The statement in Theorem 2 justifies comparing networks via their respective induced space. Similar as in Definition 6, defining $\max\{d_{\text{PE, S}}(N_{X},N_{Y}),d_{\text{PE, S}}(N_{Y},N_{X})\}$ would yield a metric in the space ${\mathcal{N}}\mod\cong$ and this maximum is the same as $d_{\text{E}}$ defined in (6). Since the induced spaces incorporate more information of the original networks while at the same time $d_{\text{PE, S}}(N_{X},N_{Y})=d_{\text{PE}}(N_{X},N_{Y})$, an approximation to $d_{\text{PE, S}}(N_{X},N_{Y})$ via the induced space would be a better approximation to $d_{\text{PE}}(N_{X},N_{Y})$. It may appear that the evaluation of the induced space is costly. However, we demonstrate in the next subsection that the partial embedding distances have a nice property that if we sample a number of points in the induced spaces respectively according to the same rule, the distance between the sampled induced space is the same as the original distance. Despite that the definition of interiors of networks is somewhat arbitrary, its practical usefulness can be justified from Theorem 2.

IV-B Sampling Of Interiors

In this section, we consider a practical scenario where we only take several samples in the induced space. We show that comparing the combination of nodes in the respective original networks and sampled nodes in the induced space would yield the same result as comparing the original networks. Given a network $N_{X}=(X,r_{X})$, our aim is to define a sampled induced space $(Q_{X},{\hat{r}}_{X})$ where $Q_{X}\supset X$ includes more nodes compared to $X$. An example is in Figure 5, where the original node space is given by $\{a,b,c\}$, and one version of sampled induced node space is $Q_{X}=\{a,b,c,e,f,g\}$, the union of the original nodes and the nodes in the midpoints of the edges in the original networks. The distance in the sampled induced space ${\hat{r}}_{X}:Q_{X}\times Q_{X}\rightarrow{\mathbb{R}}_{+}$ should preserve the distance of original points in $X$. A natural choice for ${\hat{r}}_{X}$ is the restriction of the distance $s_{X}$ defined for the induced space $S_{X}$: i.e. given any pair of points $x,x^{\prime}\in Q_{X}$, let ${\hat{r}}_{X}(x,x^{\prime}):=s_{X}(x,x^{\prime})$. Our key observation for such construction is that if the nodes in the induced spaces of a pair of networks are sampled according to the same strategy, then the distance between the sampled induced space is identical to the original distance. We start by formally describing what do we mean by a pair of networks sampled according to the same rule as next.

Definition 9

Given a pair of networks $N_{X}=(X,r_{X})$ and $N_{Y}=(Y,r_{Y})$, their respective sampled space $(Q_{X},{\hat{r}}_{X})$ and $(Q_{Y},{\hat{r}}_{Y})$ form a regular sample pair, if for any mapping $\hat{\phi}:X\rightarrow Y$ in the original node set, we have $\phi(x)\in Q_{Y}$ for any $x\in Q_{X}$, where $\phi:S_{X}\rightarrow S_{Y}$ is the map induced from $\hat{\phi}$ such that $\phi:x\mapsto\phi(x)$ whose the $i$-th element in the tuple representation $[\phi(x)]_{i}$ is

[TABLE]

and for any mapping $\hat{\psi}:Y\rightarrow X$ in the original node set, we have $\psi(y)\in Q_{X}$ for any $x\in Q_{X}$ where $\psi:S_{Y}\rightarrow S_{X}$ is the map induced from $\hat{\psi}$ such that the $j$-th element in the tuple representation of $\psi(y)$ is

[TABLE]

In the definition, $\mathbb{1}\{\hat{\phi}(j)=i\}$ is the indicator function such that it equals one if $\hat{\phi}$ maps $j\in X$ to $i\in Y$ and $\mathbb{1}\{\hat{\phi}(j)=i\}=0$ otherwise. The notation $[\phi(x)]_{i}$ denotes the proportion of $\phi(x)$ coming from $i$-th node in $Y$. It is easy to see that $\phi$ in (18) is well-defined. Firstly, $[\phi(x)]_{i}\geq 0$ for any $i\in Y$, and

[TABLE]

ensuring $\phi(x)$ is in the induced convex hull space $S_{Y}$. Secondly, for any $j\in X$ in the original nodespace, its mapping $\phi(j)$ would have the tuple representation with $[\phi(j)]_{i}=\mathbb{1}\{\hat{\phi}(j)=i\}$, a node in the original node space of $Y$. Consequently, for any $j\in X$, we have that $\phi(j)\in Y$. Combining these two observations imply that $\phi:S_{X}\rightarrow S_{Y}$ is well-defined. By symmetry, $\psi$ induced from $\hat{\psi}$ is also well-defined from $S_{Y}$ to $S_{X}$. Definition 9 states that for any point $x$ in $Q_{X}$, no matter how we relate points in $Q_{X}$ to points in $Q_{Y}$, the mapped node $\phi(x)$ should be in the induced sample space $Q_{Y}$. An example of regular sample pair is illustrated in Figure 4, where $Q_{X}=\{a,b,c,x_{1},x_{2},x_{3},x_{4},x_{5},x_{6},q\}$ is the collection of original node space and points that are combination of one-third of original nodes and $Q_{Y}=\{u,v,y_{1},y_{2}\}$. Figure 4 exemplifies the scenario for a specific mapping $\hat{\phi}$ with $\hat{\phi}(a)=\hat{\phi}(b)=u$ and $\hat{\phi}(c)=v$; it is apparent that $\phi(x)\in Q_{Y}$ for any $x\in Q_{X}$. We note that ${\tilde{Q}}_{X}=Q_{X}/\{q\}$ also form a regular sample pair with $Q_{Y}$. A pair of networks $N_{X}$ and $N_{Y}$ can be compared by evaluating their difference in their respective sampled induced space as next.

Definition 10

Given two networks $N_{X}=(X,r_{X})$ and $N_{Y}=(Y,r_{Y})$ with their respective sampled induced space $(Q_{X},{\hat{r}}_{X})$ and $(Q_{Y},{\hat{r}}_{Y})$, for a map $\phi:Q_{X}\rightarrow Q_{Y}$ such that $\phi(x)\in Y$ for any $x\in X$, define the difference with respect to $\phi$ as

[TABLE]

The partial embedding distance from $N_{X}$ to $N_{Y}$ measured with respect to the sampled induced spaces is then defined as

[TABLE]

Our key result is that $d_{\text{PE, Q}}(N_{X},N_{Y})$ is the same as the partial embedding distance $d_{\text{PE}}(N_{X},N_{Y})$ defined in (5) when the sampled space form a regular sample pair.

Theorem 3

When the sampled spaces $Q_{X}$ and $Q_{Y}$ form a regular sample pair, the function $d_{\text{PE, Q}}:{\mathcal{N}}\times{\mathcal{N}}\rightarrow{\mathbb{R}}_{+}$ defined in (21) is an embedding metric in the space ${\mathcal{N}}$. Moreover, it yields the same distance as the function $d_{\text{PE}}$ defined in (5), i.e.

[TABLE]

for any networks $N_{X}$ and $N_{Y}$.

The statement in Theorem 3 gives proper reasoning for differentiating networks via their sampled induced space. Similar as previous treatments, we could define $\max\{d_{\text{PE, Q}}(N_{X},N_{Y}),d_{\text{PE, Q}}(N_{Y},N_{X})\}$ as a metric in the space ${\mathcal{N}}\mod\cong$. Since the sampled induced spaces incorporate more information of the original networks, an approximation to $d_{\text{PE, Q}}(N_{X},N_{Y})$ via the sampled induced space would be a better approximation to $d_{\text{PE}}(N_{X},N_{Y})$. Moreover, since we can construct the sampled induced space following some predetermined strategy – taking midpoints for all edges in the networks, comparing networks via their sampled induced space is plausible in terms of complexity. Figure 5 illustrate the same network considered in Figure 1 where the multi-dimensional scaling based on the sampled induced points would succeed in distinguishing networks that are different. We illustrate the practical usefulness of such methods in the next section.

V Application

We first illustrate the usefulness of considering interiors of networks. We consider 10 networks in the form Figure 6 (a) where $\gamma=1,2,\dots,10$. Approximations of network embedding distances are evaluated. Figure 6 (b) and (c) illustrate the heat-maps of the distance approximations where the indices in both horizontal and vertical directions denote the value of $\gamma$ in the networks. When interiors are considered by adding midpoints of edges, e.g. nodes $e$, $f$, and $g$ in Figure 6 (a), network distance approximations illustrated in Figure 6 (b) yield more desired results, as networks with similar $\gamma$ are close to each other with respect to their network distance approximations. This is more apparent for networks with $\gamma\leq 5$, where the relationships in the original networks fail to satisfy triangle inequality. A detailed analysis indicates that adding interior points in the networks (i) preserve the desired property of embedding distance when interiors are not considered (the distance approximations in Figure 6 (b) and (c) are very similar for $\gamma>5$ where triangle inequalities are satisfied) and (ii) fix the undesired issue when the relationships in the original networks fail to satisfy triangle inequality.

We next consider the comparison and classification of three types of synthetic weighted networks. Edge weights in all three types of networks encode proximities. The first type of networks are with weighted Erdős-Rényi model [28], where the edge weight between any pair of nodes is a random number uniformly selected from the unit interval $[0,1]$. In the second type of networks, the coordinates of the vertices are generated uniformly and randomly in the unit circle, and the edge weights are evaluated with the Gaussian radial basis function $\exp(-d(i,j)^{2}/2\sigma^{2})$ where $d(i,j)$ is the distance between vertices $i$ and $j$ in the unit circle and $\sigma$ is a kernel width parameter. In all simulations, we set $\sigma$ to $0.5$. The edge weight measures the proximity between the pair of vertices and takes value in the unit interval. In the third type of networks, we consider that each vertex $i$ represents an underlying feature ${\mathbf{u}}_{i}\in{\mathbb{R}}^{d}$ of dimension $d$, and examine the Pearson’s linear correlation coefficient $\rho_{ij}$ between the corresponding features ${\mathbf{u}}_{i}$ and ${\mathbf{u}}_{j}$ for a given pair of nodes $i$ and $j$. The weight for the edge connecting the pair is then set as $\rho_{ij}/2+0.5$, a proximity measure in the unit interval. The feature space dimension $d$ is set as $5$ in all simulations. We want to see if network comparison tools proposed succeed in distinguishing networks generated from different processes.

We start with networks of equal size $|X|=25$ and construct $20$ random networks for each aforementioned type. We then use the multi-dimensional scaling methods introduced in [17, 18] to approximate the embedding network distance $d_{\text{E}}$ defined in Definition 6. To evaluate the effectiveness of considering interiors of networks described in Section IV, we add midpoints for all edges in a given network; it is apparent that any pair of networks with interiors defined in this way would form a regular sample pair. Approximations of the embedding network distance $d_{\text{E}}$ between these networks with midpoints added are then evaluated. Figure 7 (a) and (b) plot the two dimensional Euclidean embeddings [29] of the network metric approximations with and without interiors respectively. All embeddings in the paper are constructed with respect to minimizing the sum of squares of the inter-point distances; other common choices to minimize the sum of four power of the inter-point distances yields similar results. Networks constructed with different models form clear separate clusters (1 out of 60 errors with 1.67%) with respect to approximation of network distances between networks with interior points added, where networks with Erdős-Rényi model are denoted by red circles, networks with unit circle model are described by blue diamonds, and correlation model represented as black squares. The clustering structure is not that clear (4 out of 60 errors with 6.67%) in terms of with respect to approximation of network distances between networks without interior points, but networks constructed from different models are in general much more different compared to networks from the same model.

Next we consider networks with number of nodes ranging between $20$ and $25$. Two networks are randomly generated for each network type and each number of nodes, resulting in $60$ networks in total. Interiors are examined similarly as before by adding midpoints for all edges in a given network. Figure 7 (c) and (d) illustrate the two dimensional Euclidean embeddings of the network metric approximations with and without interiors respectively. Despite the fact that networks with same model have different number of nodes, dissimilarities between network distance approximations are smaller when their underlying networks are from the same process. Similar as in the case with same number of nodes, considering interiors result in a more distinctive clustering pattern. An unsupervised classification with two linear boundaries would yield 1 out of 60 errors ($1.67\%$) for networks with interiors added and 5 errors ($8.33\%$) without interiors.

These results illustrate that (i) comparing networks by using embedding distance succeeds in identifying networks with different properties, and (ii) adding interiors to networks to form regular sample pair as in Section IV would yield better approximations to the actual network distances. Admittedly, other methods to compare networks may also succeed in distinguishing networks, after some proper treatment towards the issue of different sizes. Nonetheless, interior and embedding method would be more universal, not only for the reason that it establishes an approximation to the actual network metrics, but also since it provides a systematic way to analyze if one network can be well matched to a subset of another network.

VI Conclusion

We present a different perspective to consider networks by defining a semimetric space induced from all the relationships in a given network. We demonstrate that comparing the respective induced space between a pair of networks outputs the identical distance as evaluating the discrepancy between the original network by embedding one network into another network, which we prove to be a valid metric in the space of all networks. Therefore, better approximations to the network metric distances can be constructed by examining the respective induced space. We illustrate that such methods succeed in classifying weighted pairwise networks constructed from different processes.

Appendix A Proofs in Section III

**Proof of Proposition 1: ** To prove that $d_{\text{PE}}$ is an embedding metric in the space of networks, we prove the (i) nonnegativity, (ii) embedding identity, and (iii) triangle inequality properties in Definition 4.

**Proof of nonnegativity property: ** Since $|r_{X}(x,x^{\prime})-r_{Y}(\phi(x),\phi(x^{\prime}))|$ is nonnegative, $\Delta_{X,Y}(\phi)$ defined in (4) also is. The partial embedding distance must then satisfy $d_{\text{PE}}(N_{X},N_{Y})\geq 0$ because it is a minimum of nonnegative numbers. ∎

**Proof of embedding identity property: ** First, we need to show that if $N_{X}$ can be isometrically embedded into $N_{Y}$, we must have $d_{\text{PE}}(N_{X},N_{Y})=0$. To see that this is true recall that for isometric embeddable networks, there exists a mapping $\phi:X\rightarrow Y$ that preserves distance functions (1). Then, under this mapping, we must have $\Delta_{X,Y}(\phi)=0$. Since $\phi$ is a particular mapping, taking a minimum over all mappings as in (5) yields

[TABLE]

Since $d_{\text{PE}}(N_{X},N_{Y})\geq 0$, it must be that $d_{\text{PE}}(N_{X},N_{Y})=0$ when $N_{X}$ can be isometrically embedded into $N_{Y}$.

Second, we need to prove $d_{\text{PE}}(N_{X},N_{Y})=0$ must imply that $N_{X}$ can be isometrically embedded into $N_{Y}$. If $d_{\text{PE}}(N_{X},N_{Y})=0$, there exists a mapping $\phi:X\rightarrow Y$ such that $r_{X}(x,x^{\prime})=r_{Y}(\phi(x),\phi(x^{\prime}))$ for any $x,x^{\prime}\in X$. This implies that $\phi$ is an isometric embedding and therefore $N_{X}$ can be isometrically embedded into $N_{Y}$. ∎

**Proof of triangle inequality: ** To show that the triangle inequality, let the mapping $\phi$ between $X$ and $Z$ and $\psi$ between $Z$ and $Y$ be the minimizing mappings in (5). We can then write

[TABLE]

Since both $\phi$ and $\psi$ are mappings, $\psi\circ\phi$ would be a valid mapping from $X$ to $Y$. The mapping $\psi\circ\phi$ may not be the minimizing mapping for the distance $d_{\text{PE}}(N_{X},N_{Y})$. However since it is a valid mapping with the definition in (5) we can write

[TABLE]

Adding and subtracting $d_{Z}(\phi(x),\phi(x^{\prime}))$ in the absolute value of \Delta_{X,Y}(\psi\circ\phi)=\max_{x,x^{\prime}\in X}\big{|}r_{X}(x,x^{\prime})-r_{Y}(\psi(\phi(x)),\psi(\phi(x^{\prime})))\big{|} and using the triangle inequality of the absolute value yields

[TABLE]

We can further bound (27) by taking maximum over each summand,

[TABLE]

The first summand in (LABEL:eqn_dfn_partial_embedding_proof_triangle_bound_2) is nothing different from $\Delta_{X,Z}(\phi)$. Since $\phi(x),\phi(x^{\prime})\in Z$, the second summand in (LABEL:eqn_dfn_partial_embedding_proof_triangle_bound_2) can be further bounded by

[TABLE]

These two observations implies that

[TABLE]

Substituting (25) and (26) into (LABEL:eqn_dfn_partial_embedding_proof_triangle_bound_4) yields triangle inequality. ∎

Having proven all statements, the global proof completes. ∎

**Proof of Theorem 1: ** To prove that $d_{\text{E}}$ is a metric in the space of networks modulo isomorphism, we prove the (i) nonnegativity, (ii) symmetry, (iii) identity, and (iv) triangle inequality properties in Definition 3.

**Proof of nonnegativity property: ** Since $d_{\text{PE}}(N_{X},N_{Y})$ as well as $d_{\text{PE}}(N_{Y},N_{X})$ are both nonnegative, the embedding distance must then satisfy $d_{\text{E}}(N_{X},N_{Y})\geq 0$. ∎

**Proof of symmetry property: ** Since $d_{\text{E}}(N_{X},N_{Y})=d_{\text{E}}(N_{Y},N_{X})=\max\{d_{\text{PE}}(N_{X},N_{Y}),d_{\text{PE}}(N_{Y},N_{X})\}$, the symmetry property follows directly. ∎

**Proof of identity property: ** First, we need to show that if $N_{X}$ and $N_{Y}$ are isomorphic, we must have $d_{\text{E}}(N_{X},N_{Y})=0$. To see that this is true recall that for isomorphic networks there exists a bijective map $\phi:X\rightarrow Y$ that preserves distance functions (1). This implies $\phi$ is also an injection, and we can find an injection $\psi:Y\rightarrow X$ that preserves distance functions (1). Then, under the injection $\phi$, we must have $\Delta_{X,Y}(\phi)=0$. Since $\phi$ is a particular mapping, taking a minimum over all mappings as in (5) yields

[TABLE]

Since $d_{\text{PE}}(N_{X},N_{Y})\geq 0$, as already shown, it must be that $d_{\text{PE}}(N_{X},N_{Y})=0$ when $N_{X}$ are isomorphic to $N_{Y}$. By symmetry we have $d_{\text{PE}}(N_{Y},N_{X})=0$, which combines with previous observation implies that $d_{\text{E}}(N_{X},N_{Y})=0$.

Second, we need to prove $d_{\text{E}}(N_{X},N_{Y})=0$ must imply that $N_{X}$ and $N_{Y}$ are isomorphic. By the definition of embedding distance, $d_{\text{E}}(N_{X},N_{Y})=0$ means $d_{\text{PE}}(N_{X},N_{Y})=0$ and $d_{\text{PE}}(N_{Y},N_{X})=0$. If $d_{\text{PE}}(N_{X},N_{Y})=0$, there exists a mapping $\phi:X\rightarrow Y$ such that $r_{X}(x,x^{\prime})=r_{Y}(\phi(x),\phi(x^{\prime}))$ for any $x,x^{\prime}\in X$. Moreover, this also implies the function $\phi$ must be injective. If it were not, there would be a pair of nodes $x\neq x^{\prime}$ with $\phi(x)=\phi(x^{\prime})=y$ for some $y\in Y$. By the definition of networks, we have that

[TABLE]

which contradicts the observation that $r_{X}(x,x^{\prime})=r_{Y}(\phi(x),\phi(x^{\prime}))$ for any $x,x^{\prime}\in X$ and shows that $\phi$ must be injective. By symmetry, simultaneously, if $d_{\text{PE}}(N_{Y},N_{X})=0$, there exists an injective mapping $\psi:Y\rightarrow X$ such that $r_{Y}(y,y^{\prime})=r_{X}(\psi(y),\psi(y^{\prime}))$ for any $y,y^{\prime}\in Y$. By applying the Cantor-Bernstein-Schroeder Theorem[30, Section 2.6] to the reciprocal injections $\phi:X\rightarrow Y$ and $\psi:Y\rightarrow X$, the existence of a bijection between $X$ and $Y$ is guaranteed. This forces $X$ and $Y$ to have same cardinality and $\phi$ and $\psi$ being bijections. Pick the bijection $\phi$ and it follows $r_{X}(x,x^{\prime})=r_{Y}(\phi(x),\phi(x^{\prime}))$ for all nodes $x,x^{\prime}\in X$. This shows that $N_{X}\cong N_{Y}$ and completes the proof of the identity statement. ∎

**Proof of triangle inequality: ** To show that the triangle inequality holds, from the definition of embedding distance, we have that

[TABLE]

Since partial embedding distance is a valid embedding metric, it satisfies triangle inequality in Definition 4, therefore, we can bound (33) by

[TABLE]

To further bound (34) we utilize the relationship as next.

Fact 1

Given real numbers $a,b,c,d$, it holds that

[TABLE]

**Proof : **If $a\geq c$ and $b\geq d$, the inequality holds since the left hand side is $a+b$ and the right hand side is also $a+b$. Similarly, if $c\geq a$ and $d\geq b$, the inequality also holds. What remains to consider are scenarios of $a\geq c,d\geq b$ as well as $c\geq a,b\geq d$. By symmetry, it suffices to consider the first scenario with $a\geq c,d\geq b$. Under this scenario, the statement becomes

[TABLE]

It follows that the state is correct following the assumption. Since we have considered all scenarios, the proof concludes. ∎

Back to the proof of triangle inequality, applying Fact 1 onto (34) yields

[TABLE]

Substituting the definition of $d_{\text{E}}(N_{X},N_{Z})$ and $d_{\text{E}}(N_{Z},N_{Y})$ into (37) yields

[TABLE]

which is the triangle inequality and completes the proof. ∎

Having proven all statements, the global proof completes. ∎

**Proof of Lemma 2: ** Denote $d^{\prime}_{\text{C}}(N_{X},N_{Y})$ to represent $\min_{\phi:X\rightarrow Y,\psi:Y\rightarrow X}\max\{\Delta_{X,Y}(\phi),\Delta_{Y,X}(\psi),\delta_{X,Y}(\phi,\psi)\}$. In order to prove prove the statement, we show that given any networks $N_{X}$ and $N_{Y}$, we have that (i) $d^{\prime}_{\text{C}}(N_{X},N_{Y})\leq d_{\text{C}}(N_{X},N_{Y})$ and that (ii) $d_{\text{C}}(N_{X},N_{Y})\leq d^{\prime}_{\text{C}}(N_{X},N_{Y})$.

**Proof of $d^{\prime}_{\text{C}}(N_{X},N_{Y})\leq d_{\text{C}}(N_{X},N_{Y})$: ** From the definition of $d_{\text{C}}(N_{X},N_{Y})$, there exists a correspondent $C$ such that $|r_{X}(x,x^{\prime})-r_{Y}(y,y^{\prime})|\leq d_{\text{C}}(N_{X},N_{Y})$ for any $(x,y),(x^{\prime},y^{\prime})\in C$. Define a function $\phi:X\rightarrow Y$ that associates $x$ with an arbitrary $y$ chosen from the set that form a pair with $x$ in $C$,

[TABLE]

Since $C$ is a correspondence the set $\{y\mid(x,y)\in C\}$ is nonempty for any $x$ implying that $\phi$ is well-defined for any $x\in X$. Hence,

[TABLE]

for any $x,x^{\prime}\in X$. Since (40) is true for any $x,x^{\prime}\in X$, it also true for the maximum pair, and therefore

[TABLE]

Define a function $\psi:Y\rightarrow X$ that associates $y$ with an arbitrary $x$ chosen from the set that form a pair with $y$ in $C$,

[TABLE]

Following the similar argument as above would yield us

[TABLE]

Finally, recall that $\delta_{X,Y}(\phi,\psi)$ is defined as $\max_{x\in X,y\in Y}|r_{X}(x,\psi(y))-r_{Y}(\phi(x),y)|$. In the same time, we have $(x,\psi(y))\in C$ as well as $(\phi(x),y)\in C$, and therefore

[TABLE]

Taking a maximum on both sides of inequlities (41), (43), and (44) yields

[TABLE]

The specific $\phi$ and $\psi$ may not be the minimizing mappings for the left hand side of (45). Nonetheless, they are valid mappings and therefore taking a minimum over all mappings yields the desired inequality $d^{\prime}_{\text{C}}(N_{X},N_{Y})\leq d_{\text{C}}(N_{X},N_{Y})$. ∎

**Proof of $d_{\text{C}}(N_{X},N_{Y})\leq d^{\prime}_{\text{C}}(N_{X},N_{Y})$: ** From the definition of $d^{\prime}_{\text{C}}(N_{X},N_{Y})$, there exists a pair of mappings $\phi:X\rightarrow Y$ and $\psi:Y\rightarrow X$ such that

[TABLE]

for any $x,x^{\prime}\in X$ and $y,y^{\prime}\in Y$. Define a correspondence by taking the union of the pairs associated by $\phi$ and $\psi$ such that

[TABLE]

Since $\phi(x)$ is defined for any $x$ and $\psi(y)$ is defined for any $y$, $C$ is a well-defined correspondence. Notice that any pair $(x,y)\in C$ in the correspondence would be one of the following two forms: $(x,\phi(x))$ or $(\psi(y),y)$. Therefore, for any pairs $(x,y),(x^{\prime},y^{\prime})\in C$, they must be from one of the following three forms (i) $(x,\phi(x)),(x^{\prime},\phi(x^{\prime}))$, (ii) $(\psi(y),y),(\psi(y^{\prime}),y^{\prime})$, or (iii) $(x,\phi(x)),(\psi(y),y)$. If they are in the form (i), from (46), we can bound the difference between the respective relationship as

[TABLE]

If the pairs are in the form (ii), (47) also implies the correctness of (50). Finally, if the pairs are in the form (iii), (50) would be established from (48). Consequently, (50) holds for any $(x,y),(x^{\prime},y^{\prime})\in C$. Therefore, they must also hold true for the bottleneck pairs achieving the maximum $\Gamma_{X,Y}(C)$ in (2) which implies that $\Gamma_{X,Y}(C)\leq d^{\prime}_{\text{C}}(N_{X},N_{Y})$. The specific correspondence $C$ may not be the minimizing one in defining $d_{\text{C}}(N_{X},N_{Y})$. Nonetheless, they are valid mappings and therefore taking a minimum over all mappings yields the desired inequality $d_{\text{C}}(N_{X},N_{Y})\leq d^{\prime}_{\text{C}}(N_{X},N_{Y})$. ∎

Since we have proven the two inequalities, it follows that $d_{\text{C}}\equiv d^{\prime}_{\text{C}}$ and this completes the proof of the statement. ∎

**Proof of Corollary 1: ** The network distance $d_{\text{C}}(N_{X},N_{Y})$ would be no smaller than the right hand side of (7), if we remove the term $\delta_{X,Y}(\phi,\psi)$ in the maximum, i.e.

[TABLE]

The right hand side of (51) would become smaller if we take the respective minimum for mappings $\phi$ and $\psi$ before taking the maximum, yielding us

[TABLE]

From (5) and (6) in Definitions 6 and 5, it is not hard to observe that the right hand side of (52) is $\max\{d_{\text{PE}}(N_{X},N_{Y}),d_{\text{PE}}(N_{Y},N_{X})\}=:d_{\text{E}}(N_{X},N_{Y})$, yielding the desired result $d_{\text{E}}(N_{X},N_{Y})\leq d_{\text{C}}(N_{X},N_{Y})$. ∎

Appendix B Proofs in Section IV

**Proof of Proposition 3: ** To prove that space $(S_{X},s_{X})$ induced from $N_{X}=(X,r_{X})$ is a semimetric space, we prove the (i) nonnegativity, (ii) symmetry, (iii) identity properties in Definition 3 and (iv) $s_{X}(p,m)=r_{X}(p,m)$ when $p,m\in X$.

**Proof of nonnegativity property: ** Since $r_{X}(i,j)>0$ for any different nodes in the original networks $i,j\in X,i\neq j$, $|w_{ij}^{\star}|r_{X}(i,j)\geq 0$ in (14). Therefore, the induced distance $s_{X}(p,m)=\sum_{i\neq j}|w_{ij}^{\star}|r_{X}(i,j)\geq 0$. ∎

**Proof of symmetry property: ** Given a pair of nodes $p,m\in S_{X}$, we would like to demonstrate that $s_{X}(p,m)=s_{X}(m,p)$. Denote $\{w_{ij}^{\star}\}$ as the collection of units of transformation along the direction from $i$ to $j$ in the original network. These vectors together make up the path from $p$ to $m$ with smallest amount of transformation. By definition, $\{w_{ij}^{\star}\}$ is the optimal solution to (13). Denote $\{v_{ij}^{\star}\}$ as the collection of units of transformation along the direction from $i$ to $j$ which makes up the path from $m$ to $p$ with the smallest amount of transformation. By definition, $\{v_{ij}^{\star}\}$ is the optimal solution to the following problem

[TABLE]

Comparing (13) with (53), it is easy to observe that if we take $v_{ij}=-w_{ij}$ for any $i<j$, the two problems becomes identical. Therefore, for the optimal solutions, we have the relationship $v_{ij}^{\star}=-w_{ij}^{\star}$ for any $i<j$. By definition in (14), this implies the two relationships are the same

[TABLE]

and completes the proof. ∎

**Proof of identity property: ** First we want to show that if $m$ and $p$ are identical points, their induced relationship $s_{X}(p,m)=0$. In such scenario, $m$ and $p$ must have same tuple representation $(m_{1},\dots,m_{n})$ and $(p_{1},\dots,p_{n})$ with $m_{i}=p_{i}$ for any $i\in X$. In this case, it is apparent that the optimal solution $\{w_{ij}^{\star}\}$ in (13) is $w_{ij}^{\star}=0$ for any $i\neq j$. Therefore, $s_{X}(p,m)=0$ shows the first part of the proof for identity property.

Second, we need to prove $s_{X}(p,m)=0$ must imply that $p$ and $m$ are the same. By definition in (14), the induced relationship can be written $s_{X}(p,m)=\sum_{i\neq j}|w_{ij}^{\star}|r_{X}(i,j)$, where the original relationship is always positive with $r_{X}(i,j)>0$ for any $i\neq j$. Therefore, $s_{X}(p,m)=0$ must imply that $|w_{ij}^{\star}|$ given any $i\neq j$. Combining this observation with the constraints in (13) imply that $p_{i}=m_{i}$ for any $i\in X$. Therefore, $p$ and $m$ are identical point in the induced space, and this completes the proof. ∎

**Proof of the property that $s_{X}(p,m)=r_{X}(p,m)$ when $p,m\in X$: ** When both $p,m\in X$, the respective tuple representation in the space is $p=(p_{1},\dots,p_{n})$ with $p_{i}=1$ if $i=p$ and $p_{i}=0$ otherwise, and $m=(m_{1},\dots,m_{n})$ with $m_{i}=1$ if $i=m$ and $m_{i}=0$. It is apparent that the path with the smallest amount of transformation from $p$ into $m$ is the exact vector from $p$ to $m$. Here we give a geometric proof using Figure 8. The induced space is the $(n-1)$-simplex with interior defined. Nodes $p$ and $m$ correspond to the vertices in the simplex with their coordinates given by the tuple representations $p=(p_{1},\dots,p_{n})$ and $m=(m_{1},\dots,m_{n})$. The problem in (13) searches for the shortest path in the simplex joining $p$ to $m$. It is then apparent that the shortest path should be the edge joining then; consequently $w_{pm}^{\star}=1$ for the edge and $w_{ij}^{\star}=0$ for any other edges $ij$. Taking this observation into (14) implies that $s_{X}(p,m)=r_{X}(p,m)$ and concludes the proof. ∎

Having proven all statements, the global proof completes. ∎

**Proof of Theorem 2: ** To prove the statements, it suffices to show $d_{\text{PE, S}}\equiv d_{\text{PE}}$. Then the fact that $d_{\text{PE, S}}$ is an embedding metric in the space ${\mathcal{N}}$ follows since $d_{\text{PE}}$ is an embedding metric in ${\mathcal{N}}$. To prove the equivalence of the pair of distances, we show that given any networks $N_{X}$ and $N_{Y}$, we have that (i) $d_{\text{PE}}(N_{X},N_{Y})\leq d_{\text{PE, S}}(N_{X},N_{Y})$ and (ii) $d_{\text{PE, S}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$.

**Proof of $d_{\text{PE}}(N_{X},N_{Y})\leq d_{\text{PE, S}}(N_{X},N_{Y})$: ** From the definition of $d_{\text{PE, S}}(N_{X},N_{Y})$, there exists a mapping $\phi:S_{X}\rightarrow S_{Y}$ in the induced space such that

[TABLE]

for any $x,x^{\prime}\in S_{X}$. Define a map $\hat{\phi}:X\rightarrow Y$ with $\hat{\phi}:i\mapsto\phi(i)$ for all $i\in X$. The map $\hat{\phi}$ is well defined since $\phi(x)\in Y$ for any $x\in X$. Proposition 3 guarantees that $r_{X}(i,j)=s_{X}(i,j)$ when $i\in X$ and $j\in X$. Therefore, we also have $r_{Y}(\hat{\phi}(i),\hat{\phi}(j))=s_{Y}(\hat{\phi}(i),\hat{\phi}(j))$. Substituting these two observations in (55) and searching for the maximum over all nodes $i,j\in X$ yields

[TABLE]

The specific $\hat{\phi}$ may not be the minimizing mapping for the left hand side of (56). Nonetheless, it is a valid mapping and therefore taking a minimum over all mappings $\phi:X\rightarrow Y$ yields the desired inequality $d_{\text{PE}}(N_{X},N_{Y})\leq d_{\text{PE, S}}(N_{X},N_{Y})$. ∎

**Proof of $d_{\text{PE, S}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$: ** From the definition of $d_{\text{PE}}(N_{X},N_{Y})$, there exists a mapping $\hat{\phi}:X\rightarrow Y$ in the original node space such that

[TABLE]

for any $i,j\in X$. Denote the cardinality of the node sets as $|X|=n$ and $|Y|=n^{\prime}$. For any $x\in S_{X}$, it has the tuple representation of $(x_{1},\dots,x_{n})$, and any $y\in S_{Y}$ possess the tuple representation of $(y_{1},\dots,y_{n^{\prime}})$. Define a map $\phi:S_{X}\rightarrow S_{Y}$ as in (18). We showed in (20) that $\phi$ is a well-defined mapping. For a given pair of points $x,x^{\prime}\in S_{X}$, denote $\{w_{ij}^{\star}\}$ as the collection of weights consisting the shortest path from $x$ to $x^{\prime}$ solving the problem in (13). By (14), the distance between $x$ and $x^{\prime}$ in the induced space $S_{X}$ is then given by $s_{X}(x,x^{\prime})=\sum_{i<j}|w_{ij}^{\star}|r_{X}(i,j)$. For the mapped pair $\phi(x)$ and $\phi(x^{\prime})$ in the induced space $S_{Y}$, we show in the next fact that the optimal path between $\phi(x)$ and $\phi(x^{\prime})$ is the optimal path between $x$ and $x^{\prime}$ mapped under $\phi$.

Fact 2

Given a pair of networks $(X,r_{X})$ and $(Y,r_{Y})$ with their respective induced space $(S_{X},s_{X})$ and $(S_{Y},s_{Y})$, for any pair of nodes $x,x^{\prime}$ in the induced space $S_{X}$, if $\{w_{ij}^{\star}\}$ is the collection of weights consisting the optimal path from $x$ to $x^{\prime}$ solving the problem in (13), then $\{w_{\phi(i)\phi(j)}^{\star}\}$ is the collection of weights consisting the optimal path from $\phi(x)$ to $\phi(x^{\prime})$ solving the problem in (13), where $w_{\phi(i)\phi(j)}^{\star}$ is the length traversed along the path parallel to the direction from $\phi(i)$ to $\phi(j)$.

**Proof : **Notice that the map $\phi$ may not be surjective; in other words, if we define ${\tilde{S}}_{Y}=\{\phi(x)\mid x\in S_{X}\}$ as the image under $\phi$, then it is likely that ${\tilde{S}}_{Y}\subsetneq S_{Y}$. We first show that the entire path of the shortest path from $\phi(x)$ to $\phi(x^{\prime})$ must lie entirely inside the space ${\tilde{S}}_{Y}$. To see this, define $n^{\prime\prime}=|\{\hat{\phi}(i)\mid i\in X\}|\subseteq Y$ as the number of unique elements in $Y$ hit by nodes in $X$ under the map $\hat{\phi}$. It then follows that ${\tilde{S}}_{Y}$ is the $(n^{\prime\prime}-1)$-simplex (convex hull) defined by vertices of $\hat{\phi}(i)$ for all $i\in X$. It follows naturally that $n^{\prime\prime}\leq n^{\prime}$. Therefore, in the language of topology, the $(n^{\prime\prime}-1)$ simplex describing ${\tilde{S}}_{Y}$ would be a face of the larger $(n^{\prime}-1)$-simplex $S_{Y}$. As an example, if $S_{Y}$ is the 3-simplex on the right of Figure 8 given by vertices $[0,1,2,3]$, then ${\tilde{S}}_{Y}$ would be the face of this 3-simplex: it may be one of the 2-simplices $[0,1,2]$, $[0,1,3]$, $[0,2,3]$, or $[1,2,3]$, or one of the 1-simplices $[0,1]$, $[0,2]$, $[0,3]$, $[1,2]$, $[1,3]$ or $[2,3]$, or one of the 0-simplices $[0]$, $[1]$, $[2]$, or $[3]$. Both $\phi(x)$ and $\phi(x^{\prime})$ are on this $(n^{\prime\prime}-1)$-simplex. It then follows geometrically that the optimal path transforming $\phi(x)$ into $\phi(x^{\prime})$ would also be on this $(n^{\prime\prime}-1)$-simplex. This implies that the entire path of the optimal path from $\phi(x)$ to $\phi(x^{\prime})$ must lie entirely inside the space ${\tilde{S}}_{Y}$.

Now, suppose the statement in Fact 2 is false, that the optimal path transforming $\phi(x)$ into $\phi(x^{\prime})$ is not given by $\{w_{\phi(i)\phi(j)}^{\star}\}$ as the collection of weights. Since this path is entirely inside the space ${\tilde{S}}_{Y}$, it needs to be of the form $\phi(x)\rightarrow\phi(k_{0})\rightarrow\phi(k_{1})\cdots\rightarrow\phi(k_{L})\rightarrow\phi(x^{\prime})$ for some $\phi(k_{0}),\dots,\phi(k_{L})\in{\tilde{S}}_{Y}$. Denote the collection of weights for this path as $\{v_{\phi(i)\phi(j)}^{\star}\}$, where $v_{\phi(i)\phi(j)}^{\star}$ is the length traversed along the path parallel to the direction from $\phi(i)\in{\tilde{S}}_{Y}$ to $\phi(j)\in{\tilde{S}}_{Y}$. The assumption that Fact 2 is false implies that

[TABLE]

Since $\phi(k_{0}),\dots,\phi(k_{L})\in{\tilde{S}}_{Y}$, we have that $k_{0},\dots,k_{L}\in S_{X}$, and therefore the path of the form $x\rightarrow k_{0}\rightarrow k_{1}\cdots\rightarrow k_{L}\rightarrow x^{\prime}$ would be a path in $S_{X}$ from $x$ to $x^{\prime}$. This path has the collection of weights as $\{v_{ij}^{\star}\}$, where $v_{ij}^{\star}$ is the length traversed along the path parallel to the direction from $i\in X$ to $j\in X$. Moreover, utilizing the relationship in (58) yields

[TABLE]

which contradicts the assumption beforehand that the optimal path from $x$ to $x^{\prime}$ is given by the collection of weights $\{w_{ij}^{\star}\}$. Therefore, it must be that the statement in Fact 2 is true and that $\{w_{\phi(i)\phi(j)}^{\star}\}$ is the collection of weights consisting the optimal path from $\phi(x)$ to $\phi(x^{\prime})$ solving the problem in (13). ∎

Back to the proof of $d_{\text{PE, S}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$, leveraging the relationship established in Fact 2, for any pair $x,x^{\prime}\in X$, the distance between their mapped points $\phi(x)$ and $\phi(x^{\prime})$ in the induced space is given by $s_{Y}(\phi(x),\phi(x^{\prime}))=\sum_{i<j}|w_{\phi(i)\phi(j)}^{\star}|r_{Y}(\phi(i),\phi(j))$. Therefore,

[TABLE]

$w_{\phi(i)\phi(j)}^{\star}$ is for notation purposes and its value is no different from $w_{ij}^{\star}$. Therefore, combining terms for each $|w_{ij}^{\star}|$ on the right hand side of in (LABEL:eqn_proof_thm_induced_space_distance_same_second_1) yields

[TABLE]

Note that $\phi(i)=\hat{\phi}(i)$ and $\phi(j)=\hat{\phi}(j)$ by construction. Substituting (57) into (61) yields an inequality as

[TABLE]

To finish the proof we further bound $\sum_{i<j}|w_{ij}^{\star}|\leq 1$ by utilizing the following important observation for the induced space.

Fact 3

Consider a network $(X,r_{X})$ with its induced space $(S_{X},s_{X})$ and $|X|=n$, for any pair of nodes $x,x^{\prime}\in S_{X}$ in the induced space with their respective tuple representation $x=(x_{1},\dots,x_{n})$ and $x^{\prime}=(x_{1}^{\prime},\dots,x_{n}^{\prime})$. Suppose that $x$ and $x^{\prime}$ share at least $n-k$ of their tuple coefficient such that for any $0\leq k\leq n$, we have $x_{l_{1}}=x^{\prime}_{l_{1}}$ for $1\leq l_{i}\leq n-k$, then if $\sum_{1\leq i\leq n-k}x_{l_{k}}=\alpha$, the optimal path $\{w_{ij}^{\star}\}$ from $x$ to $x^{\prime}$ solving the problem in (13) satisfies that

[TABLE]

**Proof : **We prove the statement by induction. The base case with $k=0$ is trivial since $x$ and $x^{\prime}$ would be the identical points. The case with $k=1$ do not exist, since $\sum_{i}x_{i}=\sum_{i}x^{\prime}_{i}=1$ and therefore we cannot have a single tuple coefficient being different. For the case with $k=2$, without loss of generality, suppose the two tuple coefficients being different for $x$ and $x^{\prime}$ are the first and second, i.e. $x=(x_{1},x_{2},x_{3},\dots,x_{n})$ and $x^{\prime}=(x_{1}^{\prime},x_{2}^{\prime},x_{3},\dots,x_{n})$, then it is apparent $x_{1}^{\prime}=x_{2}$ and $x_{2}^{\prime}=x_{1}$ and that the optimal path from $x$ to $x^{\prime}$ only involves a path from node $1$ to node $2$ with length $w_{12}=x_{1}-x_{2}\leq 1-\sum_{i=3}^{n}x_{i}$ from requirement $\sum_{i=1}^{n}x_{i}=1$.

For the induction, suppose the statement is true for $k$, we would like to show the validity of the statement for $k+1$. Without loss of generality, suppose the last $n-(k+1)$ coefficients in the tuple representations of $x$ and $x^{\prime}$ are the same. Hence, they can be represented as $x=(x_{1},\dots,x_{k},x_{k+1},x_{k+2},\dots,x_{n})$ and $x^{\prime}=(x_{1}^{\prime},\dots,x_{k}^{\prime},x_{k+1}^{\prime},x_{k+2},\dots,x_{n})$. $x$ and $x^{\prime}$ can therefore be considered as points in some $k+1$-dimensional subspace $P$, see Figure 9. Without loss of generality, we further assume that $x_{k+1}\geq x^{\prime}_{k+1}$. For any node $x^{\prime\prime}\in S_{X}$ with tuple representation $x^{\prime\prime}=(x_{1}^{\prime\prime},\dots,x_{k}^{\prime\prime},x_{k+1},\dots,x_{n})$, it shares the last $n-k$ coefficients with $x$ and therefore $x$ and $x^{\prime\prime}$ can be considered as some point in the $k$-dimensional subspace $P_{1}$. The statement holds true for $k$; consequently, there exists a path with collection of weights $\{v_{ij}^{\star}\}$ from $x$ to $x^{\prime\prime}$ solving the problem in (13) such that

[TABLE]

We argue that we can find one such $x^{\prime\prime}$ sharing $n-k$ coefficients with $x$ in the subspace $P_{1}$ such that there exists a path from $x^{\prime\prime}$ to $x^{\prime}$ that only consists of one vector parallel with the direction from the $(k+1)$-th node in $X$ to some node $i\in X$. We give a proof by geometry here: for the subspace $P_{1}$, from any points at its vertices (e.g. $b_{1}$, $c_{1}$, or $d_{1}$ in Figure 9), we could reach a corresponding point at the vertices (e.g. $b_{2}$, $c_{2}$, or $d_{2}$) of the subspace $P_{2}$ via a link parallel to the direction from node the $(k+1)$-th node in $X$ to some node $i\in X$. For example, from $b_{1}$, we can reach $b_{2}$ simply via the vector from $b_{1}$ to $b_{2}$. This means the subspace $P_{1}$ can cover the subspace $P_{2}$ by moving along the direction only consists of one vector parallel to the direction from node the $(k+1)$-th node in $X$ to some node $i\in X$. Hence, we can find one such $x^{\prime\prime}$ sharing $n-k$ coefficients with $x$ in $P_{1}$ which can reach $x^{\prime}$ via a path only consisting of one vector with length $x_{k+1}-x^{\prime}_{k+1}$ (recall we assume $x_{k+1}\geq x^{\prime}_{k+1}$). Therefore, we can construct a path with collection of weights $\{w_{ij}\}$ from $x$ to $x^{\prime}$ by transversing the path from $x$ to $x^{\prime\prime}$ with collection of weights $\{v_{ij}^{\star}\}$ followed by the line segment from $x^{\prime\prime}$ to $x^{\prime}$. This collection of path would have the property that

[TABLE]

The path from $x$ to $x^{\prime}$ via $x^{\prime\prime}$ with collection of weights $\{w_{ij}\}$ may not be the optimal path from $x$ to $x^{\prime}$, but since it is a valid path, we can safely bound $\sum_{i<j}|w_{ij}^{\star}|\leq\sum_{i<j}|w_{ij}|\leq 1-\sum_{i=k+2}^{n}x_{k}$, which shows the statement for $k+1$. This completes the induction step and therefore concludes the proof. ∎

Back to the proof of $d_{\text{PE, S}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$, considering the case with $k=n$ and $\alpha=0$ in Fact 3 yields $\sum_{ij}|w_{ij}^{\star}|\leq 1$ for any pair of nodes $x,x^{\prime}\in S_{X}$. Substituting this relationship into (62) yields

[TABLE]

Since (66) holds true for any $x,x^{\prime}\in S_{X}$, it must also be for the pair of nodes yielding the maximum discrepancy between $s_{X}(x,x^{\prime})$ and $s_{Y}(\phi(x),\phi(x^{\prime}))$; consequently, this implies $d_{\text{PE, S}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$ and concludes the proof. ∎

Since we have proven the two inequalities, it follows that $d_{\text{PE, S}}\equiv d_{\text{PE}}$ and this completes the proof of the statements. ∎

**Proof of Theorem 3: ** To prove the statements, it suffices to show $d_{\text{PE, Q}}\equiv d_{\text{PE}}$. Then the fact that $d_{\text{PE, Q}}$ is an embedding metric in the space ${\mathcal{N}}$ follows since $d_{\text{PE}}$ is an embedding metric in ${\mathcal{N}}$. To prove the equivalence of the pair of distances, we show that given any networks $N_{X}$ and $N_{Y}$ with the sample spaces form a regular sample pair, we have that (i) $d_{\text{PE}}(N_{X},N_{Y})\leq d_{\text{PE, Q}}(N_{X},N_{Y})$ and (ii) $d_{\text{PE, Q}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$.

**Proof of $d_{\text{PE}}(N_{X},N_{Y})\leq d_{\text{PE, Q}}(N_{X},N_{Y})$: ** Utilizing ${\hat{r}}_{X}(i,j)=s_{X}(i,j)=r_{X}(i,j)$ for $i,j\in X$ by the definition the sampled space, the proof follows from the proof of $d_{\text{PE}}(N_{X},N_{Y})\leq d_{\text{PE, S}}(N_{X},N_{Y})$ in the first part of proof for Theorem 2 in Appendix B. ∎

**Proof of $d_{\text{PE, Q}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$: ** From the definition of $d_{\text{PE}}(N_{X},N_{Y})$, there exists a mapping $\hat{\phi}:X\rightarrow Y$ in the original node space such that

[TABLE]

for any $i,j\in X$. Define a mapping $\tilde{\phi}:S_{X}\rightarrow S_{Y}$ induced from $\hat{\phi}$ as in (18). The fact that $Q_{X}$ and $Q_{Y}$ form a regular sample pair implies that $\phi^{\prime}(x)\in Q_{Y}$ for any $x\in Q_{X}$ and any mapping $\phi^{\prime}:Q_{X}\rightarrow Q_{Y}$. Therefore, the specific mapping $\phi:Q_{X}\rightarrow Q_{Y}$ formed by restricting $\tilde{\phi}:S_{X}\rightarrow S_{Y}$ onto $Q_{X}\subset S_{X}$ is also well-defined. In the second part of proof for Theorem 2 in Appendix B, we have demonstrated that

[TABLE]

for any pair of nodes $x,x^{\prime}\in S_{X}$ and $\tilde{\phi}:S_{X}\rightarrow S_{Y}$. Restricting (68) on pair of nodes $x,x^{\prime}\in Q_{X}\subset S_{X}$ and the mapping $\phi=\tilde{\phi}|_{Q_{X}}$ and utilizing the fact that

[TABLE]

yields the following relationship

[TABLE]

Since (70) holds true for any $x,x^{\prime}\in Q_{X}$, it must also be the case for the pairs yielding the maximum discrepancy between ${\hat{r}}_{X}(x,x^{\prime})$ and ${\hat{r}}_{Y}(\phi(x),\phi(x^{\prime}))$; hence, this implies $d_{\text{PE, Q}}(N_{X},N_{Y})\leq d_{\text{PE}}(N_{X},N_{Y})$ and completes the proof. ∎

Since we have proven the two inequalities, it follows that $d_{\text{PE, Q}}\equiv d_{\text{PE}}$ and this concludes the proof of the statements. ∎