Spectral and Dynamic Consequences of Network Specialization

TL;DR

This paper explores how network specialization influences spectral properties and stability, showing that intrinsically stable networks retain their dynamics during topological evolution, with implications for neural networks.

Contribution

It introduces a spectral analysis of network specialization effects and links intrinsic stability to the preservation of network dynamics during evolution.

Findings

Specialization affects spectral properties of networks.

Intrinsically stable networks maintain their dynamics during evolution.

Results applicable to recurrent neural networks in machine learning.

Abstract

One of the hallmarks of real networks is their ability to perform increasingly complex tasks as their topology evolves. To explain this, it has been observed that as a network grows certain subsets of the network begin to specialize the function(s) they perform. A recent model of network growth based on this notion of specialization has been able to reproduce some of the most well-known topological features found in real-world networks including right-skewed degree distributions, the small world property, modular as well as hierarchical topology, etc. Here we describe how specialization under this model also effects the spectral properties of a network. This allows us to give conditions under which a network is able to maintain its dynamics as its topology evolves. Specifically, we show that if a network is intrinsically stable, which is a stronger version of the standard notion of global stability, then the network maintains this type of dynamics as the network evolves. This is one of the first steps toward unifying the rigorous study of the two types of dynamics exhibited by networks. These are the \emph{dynamics of} a network, which is the study of the topological evolution of the network's structure, modeled here by the process of network specialization, and the \emph{dynamics on} a network, which is the changing state of the network elements, where the type of dynamics we consider is global stability. The main examples we apply our results to are recurrent neural networks, which are the basis of certain types of machine learning algorithms.

Equations

$$\beta =\{v_i,e_0,C_1,e_1,C_2,\dots ,C_m,e_m,v_j\}$$

$${\beta }_1$$

$${\beta }_3$$

$$A_{ij}=\{\begin{array}{l}\omega (e_{ij})\text{if}{e}_{ij}\in E\\ 0\text{otherwise}.\end{array}$$

$$\sigma (G)=\{\lambda \in \mathbb{C}:\det (A-\lambda I)=0\},$$

$$\rho (G)=\max \{|\lambda |:\lambda \in \sigma (A)\}.$$

$$\sigma \text{vskip}12.0ptplus4.0ptminus4.0pt({\mathcal{S}}_B(G)\text{vskip}12.0ptplus4.0ptminus4.0pt)=\sigma (G)\cup \sigma (C_1{)}^{n_1-1}\cup \sigma (C_2{)}^{n_2-1}\cup \cdots \cup \sigma (C_m{)}^{n_m-1}$$

$$\sigma (G)\approx \{2.188,1.167\pm .544i,-1.262\pm 0.238i,-1,-1,0\}.$$

$$\sigma ({\mathcal{S}}_B(G))=\sigma (G)\cup \sigma (C_1{)}^3\cup \sigma (C_2{)}^3.$$

$$In(\beta ,C_k)=\{v_i,e_0,C_1,e_1,C_2,\dots ,C_k\}\subset \beta$$

$$Out(\beta ,C_k)=\{C_k,e_k,C_{k+1},\dots ,C_m,e_m,v_j\}\subset \beta$$

$$A=\left[\begin{array}{ccccc}\underline{B}& & & & \\ Y_0& {\underline{C}}_1& & & \\ & Y_1& {\underline{C}}_2& & \\ & & \ddots & \ddots & \\ & & & Y_{k-1}& {\underline{C}}_k\end{array}\right]$$

$$T(\beta ,C_k,\lambda )=(\lambda I-{\underline{C}}_k{)}^{-1}{Y}_{k-1}(\lambda I-{\underline{C}}_{k-1}{)}^{-1}{Y}_{k-2}\cdots (\lambda I-{\underline{C}}_1{)}^{-1}{Y}_0$$

$${\mathbf{v}}_{Z_i}=T(\beta ,Z,\lambda ){\mathbf{v}}_B.$$

$${\mathbf{u}}_Z=\sum _{k=1}^{\ell }{\mathbf{v}}_{Z_k}=\sum _{k=1}^{\ell }T(\beta ,Z,\lambda ){\mathbf{v}}_B.$$

$$\mathcal{C}(Y)=\{\{Y_1,Y_2\},\{Y_3,Y_4\}\}\text{ and }\mathcal{C}(Z)=\{\{Z_1,Z_2\},\{Z_3,Z_4\}\}$$

$$\mathcal{C}(Y)=\{\{Y_1,Y_3\},\{Y_2,Y_4\}\}\text{ and }\mathcal{C}(Z)=\{\{Z_1,Z_3\},\{Z_2,Z_4\}\}$$

$${\mathbf{u}}_Z=[2.6,1.2,1.2{]}^T=[1.3,.6,.6{]}^T+[1.3,.6,.6{]}^T={\mathbf{v}}_{Z_1}+{\mathbf{v}}_{Z_3}$$

$$d_{max}(\mathbf{x},\mathbf{y})=\underset{i}{\max }{d}_i(x_i,y_i)\text{ where }\mathbf{x},\mathbf{y}\in X\text{ and }{x}_i,y_i\in X_i.$$

$$F_i=\underset{j\in I_i}{⨁}{X}_j\to X_i\text{where}{I}_i\subseteq N=\{1,2,\dots ,n\}.$$

$$\underset{k\to \infty }{\lim }{d}_{max}(F^k({\mathbf{x}}^0),{\mathbf{x}}^{\ast })=0.$$

$$F_i(\mathbf{x})=\sum _{j=1}^n{A}_{ij}{f}_{ij}(x_j),\text{for}i\in N=\{1,2,\dots ,n\}$$

$$\underline{A}={\mathcal{S}}_B(A)=\mathcal{A}({\mathcal{S}}_B(G))\in {\mathbb{R}}^{m\times m}$$

$$G_i(\mathbf{y})=\sum _{j=1}^m{\underline{A}}_{ij}{f}_{\tau (i)\tau (j)}(y_j),\text{for}i\in M=\{1,2,\dots ,m\}$$

$$R_i(\mathbf{x})=a_i{x}_i+\sum _{j=1,j\ne i}^n{W}_{ij}{g}_j(x_j)+c_i,i\in N=\{1,2,\dots ,n\}.$$

$$R\left(\begin{bmatrix}x_{1}\\ x_{2}\\ x_{3}\end{bmatrix}\right)=\left[\begin{array}[]{c}\frac{9}{10}x_{1}-2\tanh(x_{3})+\frac{7}{4}\\[1.0pt] \frac{9}{10}x_{2}+\tanh(x_{1})+\frac{5}{4}\\[1.0pt] \frac{9}{10}x_{3}+\tanh(x_{1})+\tanh(x_{2})+\frac{2}{4}\end{array}\right],$$

$$f_{ij}(x)=\{\begin{array}{ll}{g}_i(x)=\tanh (x)& \text{if}i\ne j\\ a_{i}x+c_i& \text{otherwise}\end{array}$$

$$W=\left[\begin{array}[]{ccc}0&0&-2\\ 1&0&0\\ 1&1&0\end{array}\right]$$

$$S\left(\begin{bmatrix}y_{1}\\ y_{2}\\ y_{3}\\ y_{4}\end{bmatrix}\right)=\left[\begin{array}[]{c}\frac{9}{10}y_{1}+\tanh(y_{4})+\frac{2}{4}\\[1.0pt] \frac{9}{10}y_{2}+\tanh(y_{3})+\frac{2}{4}\\[1.0pt] \frac{9}{10}y_{3}+\tanh(y_{4})+\frac{5}{4}\\[1.0pt] \frac{9}{10}y_{4}-2\tanh(y_{1})-2\tanh(y_{2})+\frac{7}{4}\\ \end{array}\right].$$

$$T\left(\begin{bmatrix}z_{1}\\ z_{2}\\ z_{3}\\ z_{4}\\ z_{5}\\ z_{6}\\ z_{7}\end{bmatrix}\right)=\left[\begin{array}[]{c}\frac{9}{10}z_{1}+\tanh(z_{4})+\tanh(z_{6})+\frac{2}{4}\\[1.0pt] \frac{9}{10}z_{2}+\tanh(z_{3})+\frac{2}{4}\\[1.0pt] \frac{9}{10}z_{3}+\tanh(z_{5})+\tanh(z_{7})+\frac{5}{4}\\[1.0pt] \frac{9}{10}z_{4}-2\tanh(z_{1})+\frac{7}{4}\\[1.0pt] \frac{9}{10}z_{5}-2\tanh(z_{1})+\frac{7}{4}\\[1.0pt] \frac{9}{10}z_{6}-2\tanh(z_{2})+\frac{7}{4}\\[1.0pt] \frac{9}{10}z_{7}-2\tanh(z_{2})+\frac{7}{4}\end{array}\right].$$

$${\Lambda }_{ij}=\underset{\mathbf{x}\in X}{\sup }\left|\frac{\partial F_i}{\partial x_j}(\mathbf{x})\right|<\infty \text{for all}i,j\in N=\{1,2,\dots ,n\}.$$

Full text

1 Introduction

Networks studied in the biological, social and technological sciences perform various tasks. How well these tasks are carried out depends both on the network’s topology, i.e. the network’s structure of interactions, as well as the dynamics of the network elements. For instance, in the biological setting neuronal networks are responsible for complicated processes related to cognition and memory, which are based on the network’s structure of connections as well as the electrical dynamics of the network neurons [5]. The function and performance of social networks such as Facebook and Twitter depend on the social interactions between individuals and on the local/global structure of established relationships. The performance of technological networks such as the internet is based both on the topology of the network’s links, e.g. connections between routers, and the router’s ability to manage requests.

A major goal of network science is to understand how a network’s topology and the dynamics of the network’s elements effects the network’s ability to carry out its function. This goal is complicated by the fact that not only are the network’s elements dynamic, i.e. have a state that changes over time, but the network’s topology also evolves over time. That is, most real-world networks are dynamic in two distinct ways.

The changing state of the network elements is referred to as the dynamics on the network. For instance, the dynamics on the World Wide Web consists of internet traffic, or how many individuals navigate between different web page over time. In a neural network the electrical behavior of the individual neurons is the dynamics on the network. In a distribution network of warehouses and delivery routes, the way in which inventory levels at each warehouse change over time describes the dynamics on the network.

In contrast, the structural evolution of a network’s topology is referred as the dynamics of the network. In a transportation network the topology of the network evolves as new roads are built to increase the number of routes between destinations. In the World Wide Web new links between web pages are added for similar reasons as are new warehouses and delivery routes in distribution networks. In other networks such as social networks the network topology evolves as individuals form new relations, etc.

In most real networks there is an interplay between the dynamics on the network and the dynamics of the network. For instance, in transportation, technological, and information networks new routes are added in response to areas of high traffic, e.g. the addition of new road in transportation systems, new routers and connections to the internet, new web pages to the World Wide Web, etc. These new routes in turn create new traffic patterns that lead to different areas of high traffic and the creation of new routes.

Determining how these two types of dynamics influence each other is an especially difficult problem. In fact, most studies consider either dynamics on the network or dynamics of the network but not the interplay of the two [27]. There are some exceptions. For instance, the area of adaptive networks uses the dynamics on the network to change or “adapt” the network’s topology [19]. However, to the best of the authors knowledge, results in this area are strictly numerical and little if anything is rigourously known. A major goal of this paper is to take steps towards unifying the rigorous analysis of the dynamics on and of networks.

A complication to achieving this objective is that there are many network growth models describing the dynamics of a network and many models describing the dynamics of a network. The most well-known of the network growth models is the Barabasi and Albert model [3] and its predecessor the Price model [31], in which elements are added one by one to a network and are preferentially attached to vertices with high degree or some variant of this process [1, 15, 24]. Other standard models include vertex copying models in which a new element is added to a network by choosing another network element uniformly at random and connecting the new element to this element’s neighbors [23, 34, 37]. The third type of network growth models are network optimization models where the topology of the network is evolved to minimize some global constraint, e.g. operating costs vs. travel times in a transportation network [17, 18].

These models are devised to create networks that exhibit some of the most widely observed features found in real networks. This includes right-skewed degree distributions, high clustering coefficients, the small-world property, etc. (see [30] for more details on these properties). Although these model are successful in a number of ways, currently, little is known regarding how the dynamics on a network is effected by the growth described in these models. Similarly, relatively little is known regarding how these models effect the spectral properties of networks. This is of interest as a network’s eigenvalues and eigenvectors are used in defining network centralities, community detection algorithms, determining network dynamics, etc. [30].

One of the hallmarks of real networks is that as their topology evolves so does their ability to perform increasingly complex versions of their function. This happens, for instance, in neural networks, which become more modular in structure as individual parts of the brain become increasingly specialized in function [35]. Similarly, gene regulatory networks can specialize the activity of existing genes to create new patterns of gene behavior [16]. In technological networks such as the internet, this differentiation of function is also observed and is driven by the need to handle and more efficiently process an increasing amount of information.

Previously, in [6] the authors introduced the specialization model of network growth built on this notion of specialization. This model is based on the idea that as a network specializes the function of one or more of its components, i.e. some subnetwork that performs a specific function, a number of copies of this component are created that are linked to the network in a way similar to the original component. These new copies “specialize” the function of the original component in that they carry out only those functions requiring these links (cf. Figure 1 and 2). Repeated application of this specialization process results in networks with a highly modular [26, 29], hierarchical [13, 22], as well as sparse topology [28, 21] each of which is a defining feature of real networks, which are not captured by previous models (see [6] for more details).

In contrast to studying the topological features produced by this model, in this paper we consider how specialization effects the spectral properties of a network. This is done by using, and in particular extending, the theory of isospectral network transformations [7, 9], which describes how certain changes in a network’s structure effect the network’s spectrum, i.e. the eigenvalues associated with the network’s weighted or unweighted adjacency matrix. We show that if a network is specialized then the spectrum of the resulting network is that of the original network together with the eigenvalues of the specialized components (see Section 3, Theorem 1). Additionally, using the theory of isospectral transformations we show how the eigenvectors of a network are effected by specialization and specifically its effect on eigenvector centrality (see Section, 3 Theorem 2)

As a network’s dynamics can be related to the network’s spectrum we can in certain cases determine how specialization of a network will effect the dynamics on the network. The type of dynamics we consider here is global stability, which is an important property in a number of systems including neural networks [10, 12, 11, 14, 36], network epidemic models [38], and in the study of congestion on computer networks [2]. The main example(s) we consider here and apply our results to are recurrent neural networks, which model the electrical activity in the brain and which form the basis of certain algorithms in machine learning [33].

In a stable dynamical network the network always evolves to the same unique state irrespective of the network’s initial state. We show that if a dynamical network is intrinsically stable, which is a stronger form of a standard notion of stability (see Definition 11 or [8] for more details), then any specialized version of this network will also be intrinsically stable (see Section 4, Theorem 4). Hence, network growth via specialization will not destabilize the network’s dynamics if the network has this stronger version of stability. This is potentially useful in real-world applications since network growth can have a destabilizing effect on a network. For instance, a well-known example of this phenomena is cancer, which is the abnormal growth of cells that can impair the function and ultimately lead to the failure of the system.

Although networks exhibit many other types of dynamics including multistability, periodicity, and synchronization the reason we study stability is because of its relative simplicity and use it as a first step towards rigourously understanding the interplay of network dynamics and network growth. Importantly, our results suggest that if a network’s growth is due to specialization and the network has a “strong” form of dynamics, e.g. intrinsic stability, the network can maintain its dynamics as its structure evolves. It is worth noting that the ability to maintain dynamics during periods of growth is an important feature found in real networks, e.g. the cellular network of a beating heart maintain’s its periodic beating as an adolescent heart grows into an adult heart.

This paper is organized as follows. In Section 2 we describe the specialization model of network growth. In Section 3 we give our main results regarding the spectral properties of specialized networks, which describe the effect of a specialization on the eigenvalues and eigenvectors associated with a network or graph. In Section 4 we adapt the method of graph specializations to evolve the structure of a general class of dynamical systems used to model network dynamics, which include recurrent neural networks. We show that if such dynamical systems, which we refer to as dynamical networks, are intrinsically stable then any specialized version of the network is also intrinsically stable. Section 5 discusses a number of variants of the specialization method introduced in [6] and described in Section 2 as well as their spectral and dynamic consequences to a network. Proofs of the paper’s main results together with the necessary parts of the theory of isospectral network transformations are placed in Section 6. The last section, Section 7 contains some concluding remarks.

2 The Specialization Process

The topology of a network, which is the network’s structure of interactions, is most often represented by a graph. A graph $G=(V,E,\omega)$ is composed of a vertex set $V$, an edge set $E$, and a function $\omega$ used to weight the edges of the graph. The vertex set $V$ represents the elements of the network, while the edges $E$ represent the links or interactions between these network elements. The weights of the edges, given by $\omega$, typically give some measure of the strength of these interactions. Here we consider weights that are real numbers, which account for the vast majority of weights used in network analysis [30].

For the graph $G=(V,E,\omega)$ we let $V=\{v_{1},\dots,v_{n}\}$, where $v_{i}$ represents the $i$th network element. An edge between vertices $v_{i}$ and $v_{j}$ can be either directed or undirected depending on the particular network. In the undirected case in which an interaction is reciprocal, e.g. a friendship in a social network, one can consider an undirected edge to be two directed edges: one edge pointing from the first to the second element, the other pointing from the second to the first element. We can, therefore, consider any graph to be a directed graph. For a graph $G=(V,E,\omega)$ we let $e_{ij}$ denote the directed edge that begins at $v_{j}$ and ends at $v_{i}$. In terms of the network, the edge $e_{ij}$ belongs to the edge set $E$ if the $j$th network element has some direct influence or is linked to the $i$th network element in some way.

The edges $E$ of a graph can also either be weighted or unweighted. If the edges of a graph $G$ are unweighted we write $G=(V,E)$. However, any unweighted graph can be considered weighted by giving each edge unit weight. The class of graphs we consider, without any loss in generality, in this paper are weighted directed graphs. To motivate the specialization model introduced in [6] of a graph $G$, representing some network, we consider the following example of disambiguation in a Wikipedia network, a subset of the World Wide Web.

Example 2.1**.**

(Wikipedia Disambiguation) The website Wikipedia is a collection of webpages consisting of articles that are linked by topic. The website evolves as new articles are either added, linked, and/or modified within the existing website. One of the ways articles are modified is via the process of disambiguation. That is, if an article’s content is deemed to refer to a number of distinct topics then the article can be disambiguated by separating the article into a number of new articles, each on a more specific or specialized topic than the original.

Wikipedia’s own page on disambiguation gives the example that the word “Mercury” can refer to either Mercury the mythological figure, Mercury the planet, or mercury the element. To emphasize these differences, the Wikipedia page on Mercury has since been disambiguated into three pages; one for each of these subcategories. Users arriving at the Wikipedia “Mercury” page from some related web page are redirected to these pages. The result of this disambiguation is shown in Figure 1. In the original undifferentiated Mercury page users arriving from other pages could presumably find links to other mythology, planet and element pages (see Figure 1, left). After the page was disambiguated users were linked to only those relevant to the particular “Mercury”, e.g. mythology, planet, or element (see Figure 1, right). In terms of the topology of the network, this disambiguation results in the creation of a number of new “Mercury” pages each of which is linked to a subset of pages that were linked to the original Mercury page.

In this example, growth via disambiguation is a result of the number of new copies of the original webpage. However, what is important to the functionality of the new specialized network is that the way in which these new copies are linked to the unaltered pages reflects the topology of the original network. In the specialization model the way in which we link these new components is by separating out the paths and cycles on which these components lie, in a way that mimics the original network structure.

To describe the model of network specialization introduced in [6] and its spectral and dynamic consequences we first need to describe the paths and cycles of a graph. A path $P$ in the graph $G=(V,E,\omega)$ is an ordered sequence of distinct vertices $P=v_{1},\dots,v_{m}$ in $V$ such that $e_{i+1,i}\in E$ for $i=1,\dots,m-1$. If the first and last vertices $v_{1}$ and $v_{m}$ are the same then $P$ is a cycle. If it is the case that a cycle contains a single vertex then we call this cycle a loop.

Another fundamental concept that we require is the notion of a strongly connected component. A graph $G=(V,E,\omega)$ is strongly connected if for any pair of vertices $v_{i},v_{j}\in V$ there is a path from $v_{i}$ to $v_{j}$ or, in the trivial case, $G$ consists of a single vertex. A strongly connected component of a graph $G$ is a subgraph that is strongly connected and is maximal with respect to this property.

Because we are concerned with evolving the topology of a network in ways that preserve, at least locally, the network’s topology we will also need the notion of a graph restriction. For a graph $G=(V,E,\omega)$ and a subset $B\subseteq V$ we let $G|_{B}$ denote the restriction of the graph $G$ to the vertex set $B$, which is the subgraph of $G$ on the vertex set $B$ along with any edges of the graph $G$ between the vertices in $B$. We let $\bar{B}$ denote the complement of $B$, so that the restriction $G|_{\bar{B}}$ is the graph restricted to the complement of vertices in $B$.

The key to specializing the structure of a graph is to look at the strongly connected components of the restricted graph $G|_{\bar{B}}$. If $C_{1},\dots,C_{m}$ denote these strongly connected components then we need the collection of paths or cycles of these components, which we refer to as components branches.

Definition 1**.**

**(Component Branches) For a graph $G=(V,E,\omega)$ and vertex set $B\subseteq V$ let $C_{1},\dots,C_{m}$ be the strongly connected components of $G|_{\bar{B}}$. If there are edges $e_{0},e_{1},\dots,e_{m}\in E$ and vertices $v_{i},v_{j}\in B$ such that

(i) $e_{k}$ is an edge from a vertex in $C_{k}$ to a vertex in $C_{k+1}$ for $k=1,\dots,m-1$;

(ii) $e_{0}$ is an edge from $v_{i}$ to a vertex in $C_{1}$; and

(iii) $e_{m}$ is an edge from a vertex in $C_{m}$ to $v_{j}$, then we call the ordered set**

[TABLE]

a path of components in $G$ with respect to $B$. If $v_{i}=v_{j}$ then $\beta$ is a cycle of components. We call the collection $\mathcal{B}_{B}(G)$ of these paths and cycles the component branches of $G$ with respect to the base set of vertices $B$.

The sequence of components $C_{1},\dots,C_{m}$ in this definition can be empty in which case $m=0$ and $\beta$ is the trivial path $\beta=\{v_{i},v_{j}\}$ or loop if $v_{i}=v_{j}$. It is worth emphasizing that each branch $\beta\in\mathcal{B}_{B}(G)$ is a subgraph of $G$. Consequently, the edges of $\beta$ inherit the weights they had in $G$ if $G$ is weighted. If $G$ is unweighted then its component branches are likewise unweighted.

Once a graph has been decomposed into its various branches we construct the specialized version of the graph by merging these branches as follows.

Definition 2**.**

(Graph Specialization) Suppose $G=(V,E,\omega)$ and $B\subseteq V$. Let $\mathcal{S}_{B}(G)$ be the graph which consists of the component branches $\mathcal{B}_{B}(G)=\{\beta_{1},\dots,\beta_{\ell}\}$ in which we merge, i.e. identify, each vertex $v_{i}\in B$ in any branch $\beta_{j}$ with the same vertex $v_{i}$ in any other branch $\beta_{k}$. We refer to the graph $\mathcal{S}_{B}(G)$ as the specialization of $G$ over the base vertex set $B$.

A specialization of a graph $G$ over a base vertex set $B$ is a two step process. The first step is the construction of the graph’s component branches. The second step is the merging of these components into a single graph. We note that, in a component branch $\beta\in\mathcal{B}_{B}(G)$ only the first and last vertices of $\beta$ belong to the base $B$. The specialized graph $\mathcal{S}_{B}(G)$ is therefore the collection of branches $\mathcal{B}_{B}(G)$ in which we identify an endpoint of two branches if they are the same vertex. This is demonstrated in the following example.

Example 2.2**.**

**(Constructing Graph Specializations) Consider the unweighted graph $G=(V,E)$ shown in Figure 2 (left). For the base vertex set $B=\{v_{1},v_{2}\}$, which are shown in red in the figure once on the left and once on the right, the specialization $\mathcal{S}_{B}(G)$ is constructed as follows.

**

Step 1:* Construct the branch components of $G$ with respect to $B$. The graph $G|_{\bar{B}}$ has the strongly connected components $C_{1}=G|_{\{v_{3},v_{4},v_{5}\}}$ and $C_{2}=G|_{\{v_{6},v_{7},v_{8}\}}$, which are the green and blue vertices in Figure 2 (right), respectively. The set $\mathcal{B}_{B}(G)$ of all paths and cycles of components beginning and ending at vertices in $B$ consists of the component branches*

[TABLE]

which can be seen in Figure 2 (right) along with the trivial branches $\{v_{1},v_{2}\}$ and ${v_{2},v_{1}}$.

Step 2:* Merge the branch components. By merging each of the vertices $v_{1}\in B$ in all branches of $\mathcal{B}_{B}(G)=\{\beta_{1},\beta_{2},\beta_{3},\beta_{4}\}$ and doing the same for the vertex $v_{2}\in B$, the result is the graph $\mathcal{S}_{B}(G)$ shown in Figure 2 (right), which is the specialization of $G$ over the base vertex subset $B$.*

Essentially, after selecting a base set $B\subseteq V$ from a graph (network), the specialization model creates a new graph (network) consisting of all the component branches $\mathcal{B}_{B}(G)$, separated out from one another. Thus a copy of each strongly connected components of $G|_{\bar{B}}$ will appear as a copy in the new graph for each component branch which includes it. In this sense the strongly connected components $C_{1},\dots,C_{m}$ are specialized as the network evolves under this process.

Once a graph has been specialized it can again be specialized by choosing a new base of the specialized graph. In this way a network can be sequentially specialized. As a simple example one can randomly choose a fixed percentage of the graph’s vertices at each step (see Figure 3). The result is a graph that has many features consistent with real-world networks. For example, it has a right-skewed degree distribution, is disassociative, has the small world property, is sparse, and its topology is both modular and hierarchical (see Example 3.1 in [6]). As one might expect, the resulting sequence of specializations depends very much on the way in which a base is chosen at each step in the specialization process (cf. Example 3.2 and 3.3 in [6]).

In the following section we consider how the process of specialization effects the spectral properties of a network.

3 Spectral Properties of Specializations

To understand how specializing a network’s topology effects the network’s dynamics and in turn the network’s function, we need some notion that relates both structure and dynamics. One of the most fundamental concepts that relates the two is the notion of a network’s spectrum [39, 7, 8]. Spectral properties are used in a number of network applications including determining network centralities and network communities [30], which will be important in this section where we describe specializations’s effect on a network’s eigenvector centrality.

The spectrum of a network can be defined in a number of ways since a number of matrices can be associated with a network. This includes various Laplacian matrices, distance matrices, and adjacency matrices of a graph. The type of matrix we consider here is the weighted adjacency matrix of a graph. The weighted adjacency matrix of a graph $G=(V,E,\omega)$ is the matrix $A=\mathcal{A}(G)$ where

[TABLE]

If $G$ is unweighted then each entry $A_{ij}=\omega(e_{ij})=1$ if $e_{ij}\in E$ and $A_{ij}=0$ otherwise. The eigenvalues of the matrix $A\in\mathbb{R}^{n\times n}$ make up the graph’s spectrum, which we denote by

[TABLE]

where we consider $\sigma(G)$ to be a set that includes multiplicities, i.e. a multiset. The spectral radius of $G$ is the spectral radius of $A=\mathcal{A}(G)$ denoted by

[TABLE]

In Section 4 we will investigate the connection between the spectrum of a graph $G$ and the dynamics of the network associated with it.

Because we are concerned with the spectrum of a graph, which is a set that includes multiplicities, the following is important for our discussion. First, the element $\alpha$ of the set $A$ has multiplicity $m$ if there are $m$ elements of $A$ equal to $\alpha$. If $\alpha\in A$ with multiplicity $m$ and $\alpha\in B$ with multiplicity $n$ then

(i) the union $A\cup B$ is the set in which $\alpha$ has multiplicity $m+n$; and

(ii) the difference $A-B$ is the set in which $\alpha$ has multiplicity $m-n$ if $m-n>0$ and where $\alpha\notin A-B$ otherwise.

For ease of notation, if $A$ and $B$ are sets that include multiplicity then we let $B^{k}=\cup_{i=1}^{k}B$ for $k\geq 1$. That is, the set $B^{k}$ is $k$ copies of the set $B$ where we let $B^{0}=\emptyset$. For $k=-1$ we let $A\cup B^{-1}=A-B$. With this notation in place, the spectrum of a graph $G$ and the spectrum of the specialized graph $\mathcal{S}_{B}(G)$ are related by the following result.

Theorem 1**.**

(Spectra of Specialized Graphs) Let $G=(V,E,\omega)$, $B\subseteq V$, and let $C_{1},\dots,C_{m}$ be the strongly connected components of $G|_{\bar{B}}$. Then

[TABLE]

where $n_{i}$ is the number of copies of the component $C_{i}$ in $\mathcal{S}_{B}(G)$.

If a network has an evolving structure that can be modeled via a graph specialization, or more naturally a sequence of specializations, then Theorem 1 allows us to effectively track the changes in the network’s spectrum, resulting from the components $C_{1},\dots,C_{m}$. Specifically, the eigenvalues of the resulting specialized graph $G$ are the eigenvalues of the original unspecialized graph together with the eigenvalues of the new copies of the strongly connected components $C_{1},\dots,C_{m}$ appearing in $\mathcal{S}_{B}(G)$.

In nearly every network $G=(V,E,\omega)$ edges are considered to have positive edge weights, i.e. $\omega:E\rightarrow\mathbb{R}^{+}$ (see [Newman]). In this situation Theorem 1 has the following corollary, which is fundamental to the results regarding network dynamics in Section 4.

Corollary 3.3**.**

(Spectral Radius of Specialized Graphs) Suppose $G=(V,E,\omega)$ has positive edge weights. Then for any $B\subseteq V$ the spectral radius $\rho(\mathcal{S}_{B}(G))=\rho(G)$.

Proof.

Suppose that $G=(V,E,\omega)$ is a graph with positive edge weights and $B\subseteq V$ is a base. Note that as any strongly connected component $C_{i}$ of $G|_{\bar{B}}$ is a subgraph of $G$ then, as $G$ has positive edge weights, the spectral radius $\rho(C_{i})\leq\rho(G)$ (see [20]). It then follows directly from Theorem 1 that $\rho(\mathcal{S}_{B}(G))=\rho(G)$. ∎

Hence, if a graph has positive weights then its spectral radius is preserved under specialization or any sequence of specializations. The reason is that by construction the edge weights of a graph are preserved when the graph is specialized (see Definition 2). Hence, if a graph $G$ has positive edge weights then so do any of its specializations or sequential specializations.

We defer the proof of Theorem 1 until Section 6. For now, we consider an example of Theorem 1.

Example 3.4**.**

The unwieghted graph $G=(V,E)$ in Figure 2 (left) has eigenvalues

[TABLE]

Note that for $B=\{v_{1},v_{2}\}$ the graph $G|_{\bar{B}}$ has the strongly connected components $C_{1}$ and $C_{2}$ with eigenvalues $\sigma(C_{1})=\{2,-1,-1\}$ and $\sigma(C_{2})=\{\pm\sqrt{2},0\}$. Since each of the nontrivial branches of $\mathcal{B}_{B}(G)$ contain one copy of $C_{1}$ and $C_{2}$ respectively, Theorem 1 implies that the specialized graph $\mathcal{S}_{B}(G)$ has the twenty-six eigenvalues

[TABLE]

Moreover, by inspecting the eigenvalues of $G$, $C_{1}$, and $C_{2}$ it follows from Equation 2 that $\rho(S_{B}(G))=\rho(G)\approx 2.188$. This follows from Corollary 3.3 as the graph $G$ has unit edge weights, which are positive.

Not only are the eigenvalues and spectral radius of a graph $G$ preserved, in a specific way, as the graph is specialized but so are its eigenvectors. Here an eigenvector of a graph $G$ corresponding to the eigenvalue $\lambda\in\sigma(G)$ is a vector $\mathbf{x}$ such that $\mathcal{A}(G)\mathbf{x}=\lambda\mathbf{x}$, in which case $(\lambda,\mathbf{x})$ an eigenpair of $G$. That is, the eigenvalues of $G$ are the eigenvalues of its adjacency matrix $A=\mathcal{A}(G)$.

To describe how specialization effects the eigenvectors of a network we require the following definition.

Definition 3**.**

(Incoming and Outgoing Component Branches) For the graph $G=(V,E,\omega)$ and base $B\subseteq V$ let $\beta=\{v_{i},e_{0},C_{1},e_{1},C_{2},\dots,C_{m},e_{m},v_{j}\}\in\mathcal{B}_{B}(G)$. We call the ordered set

[TABLE]

the incoming branch of $\beta$ up to $C_{k}$. Similarly, we call the ordered set

[TABLE]

the outgoing branch in $\beta$ from $C_{k}$.

If $Z$ is a strongly connected component of $G|_{\bar{B}}$ then $\ell\geq 0$ copies of it will appear in the graph $\mathcal{S}_{B}(G)$, which we denote by $\mathcal{C}(Z)=\{Z_{1},Z_{2},\dots,Z_{\ell}\}$. Here, each $Z_{i}$ is associated with the component $Z$ in a single branch $\beta_{i}\in\mathcal{B}_{B}(G)$. We say $Z_{i},Z_{j}\in\mathcal{C}(Z)$ have the same incoming branch if $In(\beta_{i},Z)=In(\beta_{j},Z)$ and the same outgoing branch if $Out(\beta_{i},Z)=Out(\beta_{j},Z)$.

Definition 4**.**

(Eigenvector Transfer Matrix) For $\beta=\{v_{i},e_{0},C_{1},e_{1},C_{2},\dots,C_{m},e_{m},v_{j}\}\in\mathcal{B}_{B}(G)$ let the adjacency matrix of the incoming branch $In(\beta,C_{k})$ together with $G|_{B}$ be the matrix

[TABLE]

where $\underline{B}=\mathcal{A}(G|_{B})$ and $\underline{C}_{i}=\mathcal{A}(C_{i})$. We call the matrix

[TABLE]

the eigenvector transfer matrix of $In(\beta,C_{k})$, where $\lambda$ is a spectral parameter.

If $\mathbf{x}$ is an eigenvector of $G$ and $S\subseteq V$ is any subset of its vertex set, we let $\mathbf{x}_{S}$ denote the vector $\mathbf{x}$ restricted to the entries indexed by $S$. Similarly, by slight abuse of this notation, if $Z$ is a subgraph of $G$ with vertex set $S$ we let $\mathbf{x}_{Z}=\mathbf{x}_{S}$. With this in place we state the following theorem describing how the eigenvectors of a graph are effected by specialization.

Theorem 2**.**

**(Eigenvectors of Specialized Graphs) Let $G=(V,E,\omega)$ be a graph, $B\subseteq V$ a base, and let $Z$ be a strongly connected component of $\beta\in\mathcal{B}_{B}(G)$. If $(\lambda,\mathbf{u})$ is an eigenpair of $G$ with $\lambda\notin\sigma(G|_{\bar{B}})$ then there is an eigenpair $(\lambda,\mathbf{v})$ of $\mathcal{S}_{B}(G)$ such that the following hold:

(i) $\mathbf{u}_{B}=\mathbf{v}_{B}$.

(ii) For all $Z_{i}\in\mathcal{C}(Z)$ the eigenvector restriction**

[TABLE]

*Hence, if $Z_{i},Z_{j}\in\mathcal{C}(Z)$ have the same incoming branch then $\mathbf{v}_{Z_{i}}=\mathbf{v}_{Z_{j}}$.

(iii) For $Z_{i}\in\mathcal{C}(Z)$ let $\cup_{k=1}^{\ell}\{Z_{k}\}$ be the copies of $Z$ that have the same outgoing branch as $Z_{i}$. Then*

[TABLE]

Part (i) of Theorem 2 states that the graphs $G$ and $\mathcal{S}_{B}(G)$ have the same eigenvectors if we restrict our attention to those entries that correspond to the base vertices $B$ and to those eigenvectors with eigenvalues in $\sigma(G)-\sigma(G|_{\bar{B}})\subset\sigma(\mathcal{S}_{B}(G))$. Part (ii) states that the eigenvectors associated with the component $Z_{i}$ can be found by applying the transfer matrix $T(\beta,Z,\lambda)$ to the same eigenvector associated with $G|_{B}$. Hence, if two specializations $Z_{i}$ and $Z_{j}$ of the same component $Z$ have the same incoming branch, their associated eigenvectors are identical. Part (iii) states that if we sum the eigenvectors associated of each copy $\cup_{k=1}^{\ell}\{Z_{k}\}$ of $Z$ with the same outgoing branch in $\mathcal{S}_{B}(G)$ then we recover the eigenvector associated with the original component $Z$ in $G$.

Example 3.5**.**

Consider the unweighted graph $G=(V,E)$ in Figure 4 and its specialization over the base $B=\{v_{1},v_{2}\}$, shown in red, as in Example 2.2. Here, for simplicity, we let the components $C_{1}=Y$ and $C_{2}=Z$, in which case $\mathcal{C}(Y)=\{Y_{1},\dots,Y_{4}\}$ and $\mathcal{C}(Z)=\{Z_{1},\dots,Z_{4}\}$ are the copies of $Y$ and $Z$, respectively, in $\mathcal{S}_{B}(G)$. Note that

[TABLE]

are the partition of $\mathcal{C}(Y)$ and $\mathcal{C}(Z)$ into components with the same incoming branches and

[TABLE]

are the partition of these sets into components with the same outgoing branches.

In Figure 4, numbers next to vertices in both $G$ and $\mathcal{S}_{B}(G)$ are the associated entries of the graph’s eigenvectors $\mathbf{u}$ and $\mathbf{v}$ corresponding to the spectral radius $\rho$ of both graphs, respectively. Since $\rho=\rho(G)\approx 2.188$ is an eigenvalue of $G$ that is not an eigenvalue of $G|_{\bar{B}}$, part (i) of Theorem 2 implies that $\mathbf{u}_{B}=[1,1]^{T}=\mathbf{u}_{B}$ as can be seen in the figure. Also, as $Z_{1}$ and $Z_{2}$ have the same incoming branch in the specialized graph, part (ii) of Theorem 2 implies that $\mathbf{v}_{Z_{1}}=[1.3,.6,.6]^{T}=\mathbf{v}_{Z_{2}}$. The same can be quickly verified for the other components with the same incoming branches. Last, since $Z_{1}$ and $Z_{3}$ are the copies of $Z$ with the same outgoing branches then part (iii) of Theorem 2 implies

[TABLE]

and the same can be verified for the other components with identical outgoing branches. Additionally, the eigenvector transfer matrix can be used to determine any of the component’s associated eigenvectors in the specialized graph using only this matrix and the vector $\mathbf{v}_{B}=[1,1]^{T}$ so long as the eigenvalues associated with the eigenvector is not an eigenvalue of $G|_{\bar{B}}$.

One immediate consequence of Theorem 2 is that we can describe to a large extent what happens to the eigenvector centrality of a network as the network becomes increasingly specialized. Eigenvector centrality is one of the standard network measures used for determining the importance of a vector in a network (see [30] for more details). For an unweighted graph $G=(V,E)$ that is strongly connected the graphs eigenvector $\mathbf{p}$ corresponding to its spectral radius gives the relative ranking $p_{i}$ to each vertex $v_{i}\in V$. This value $p_{i}$ is referred to as the eigenvector centrality of the vertex $v_{i}$. Here we refer to the vector $\mathbf{p}$ as an eigencentrality vector of the graph $G$.

Corollary 3.6**.**

**(Eigenvector Centrality of Specialized Networks) Let $G=(V,E)$ be an unweighted strongly connected graph with base $B\subseteq V$ and $Z$ a strongly connected component of $G|_{\bar{B}}$ . If $\mathbf{p}$ is the eigencentrality vector of $G$, there exists an eigencentrality vector $\mathbf{q}$ of $\mathcal{S}_{B}(G)$ such that

(i) $\mathbf{p}_{B}=\mathbf{q}_{B}$;

(ii) if $Z_{i},Z_{j}\in\mathcal{C}(Z)$ have the same incoming branch then $\mathbf{q}_{Z_{i}}=\mathbf{q}_{Z_{j}}$; and

(iii) if $\cup_{k=1}^{\ell}\{Z_{k}\}$ are the copies of $Z$ that have the same outgoing branch as some $Z_{i}\in\mathcal{C}(Z)$ then $\mathbf{p}_{Z}=\sum_{k=1}^{\ell}\mathbf{q}_{Z_{k}}$.**

Proof.

By convention, the unweighted graph $G=(V,E)$ has positive edge weights as each edge is assigned unit weight. Since $G$ is strongly connected, the Perron-Frobenius theorem implies that the spectral radius $\rho=\rho(G)$ is a simple eigenvalue of $G$ with corresponding eigencentrality vector $\mathbf{p}$. Since $G|_{\bar{B}}$ is a subgraph of the strongly connected graph $G$ then the spectral radius $\rho(G|_{\bar{B}})<\rho(G)$ (see [20]). Hence, $\rho\notin\sigma(G|_{\bar{B}})$. Part (i) of Theorem 2 then implies that there is an eigenpair $(\rho,\mathbf{q})$ of $\mathcal{S}_{B}(G)$ such that $\mathbf{p}_{B}=\mathbf{q}_{B}$.

Note that $S_{B}(G)$ is both unweighted and strongly connected as $G$ is unweighted and strongly connected. By Corollary 3.3 both $G$ and $\mathcal{S}_{B}(G)$ have the same spectral radius $\rho$. Therefore, $\mathbf{q}$ must be an eigencentrality vector of $\mathcal{S}_{B}(G)$, completing the proof of part (i).

Parts (ii) and (iii) then follow immediately from parts (ii) and (iii) of Theorem 2. ∎

When a network is specialized the eigenvector centrality of the unspecialized vertices, i.e. the vertices in the base, stay the same. In contrast, this process does effect the centrality of those vertices that are specialized (cf. Figure 4). What we can say for these vertices is that if two vertices are copies of the same vertex that belong to components with the same incoming branches then these vertices have the same eigenvector centrality. Also, if we take all the specializations, i.e. copies, of a vertex that belong to components with the same outgoing branch then summing these vertices’ centralities results in the centrality of the original vertex. That is, the centrality of a vertex is distributed among this set of its copies (cf. Figure 4).

As with the proof of Theorem 1, the proof of Theorem 2 is given in Section 6. These proofs rely on a number of results from the theory of isospectral graph reductions found in [7, 9], which can also be found in Section 6.

4 Network Growth, Dynamics, and Function

Sections 2 and 3 of this paper are primarily concerned with the dynamics of a network, i.e. the temporal evolution of the network’s structure of interactions, and the spectral consequences of this evolution. This evolution also effects the network’s ability to perform its intended function. This function is not only dependent on the network’s topology but also on the type of dynamics that emerges from the interactions between the network elements, i.e. the dynamics on the network. For instance, power is transferred efficiently in power grids when the grid is synchronized.

In this section our goal is to understand how the dynamics of a network can impact the dynamics on the network. Specifically, we study under what condition(s) a network can maintain its dynamics as the network’s structure evolves via specialization. This ability to maintain functionality even as the network grows is observed in many systems. A prime example is the physiological network of organs within the body, all of which develop over time but maintain specific functions [4].

The dynamics on a network with a fixed structure can be formalized as follows.

Definition 5**.**

(Dynamical Network)* Let $(X_{i},d_{i})$ be a complete metric space where $X_{i}\subseteq\mathbb{R}$ and let $(X,d_{max})$ be the complete metric space formed by giving the product space $X=\bigoplus_{i=1}^{n}X_{i}$ the metric*

[TABLE]

Let $F:X\to X$ be a $C^{1}(X)$ map with $i^{th}$ component function $F_{i}:X\to X_{i}$ given by

[TABLE]

The discrete-time dynamical system $(F,X)$ generated by iterating the function $F$ on $X$ is called a dynamical network. If an initial condition $\mathbf{x}^{0}\in X$ is given, we define the $k^{th}$ iterate of $\mathbf{x}^{0}$ as $\mathbf{x}^{k}=F^{k}(\mathbf{x}^{0})$, with orbit $\{F^{k}(\mathbf{x}^{0})\}_{k=0}^{\infty}=\{\mathbf{x}^{0},\mathbf{x}^{1},\mathbf{x}^{2},\ldots\}$ in which $\mathbf{x}^{k}$ is the state of the network at time $k\geq 0$.

The component function $F_{i}$ describes the dynamics of the $i^{th}$ network element that emerges from its interactions with a subset of the other network elements where there is a directed interaction between the $i^{th}$ and $j^{th}$ elements if $j\in I_{i}$. For the initial condition $\mathbf{x}^{0}\in X$ the state of the $i^{th}$ element at time $k\geq 0$ is $x^{k}_{i}=(F^{k}(\mathbf{x}^{0}))_{i}\in X_{i}$ where $X_{i}$ is the state space of the $i^{th}$ element. The state space $X=\bigoplus_{i=1}^{n}X_{i}$ is the collective state space of all network elements.

For simplicity in our discussion we assume that the map $F:X\rightarrow X$ is continuously differentiable and that each $X_{i}$ is some closed interval of real numbers (see Definition 5). In fact, all of our results in this section hold in the more general setting where $F:X\rightarrow X$ is Lipschitz continuous and $X$ is any complete metric space (see [7, 9]).

The specific type of dynamics we consider here is global stability, which is observed in a number of systems including neural networks [10, 12, 11, 14, 36], in epidemic models [38], and is also important in the study of congestion in computer networks [2]. In a globally stable network, which we will simply refer to as stable, the state of the network tends towards an equilibrium irrespective of its present state. Formally, network stability is defined as follows.

Definition 6**.**

(Network Stability) The dynamical network $(F,X)$ is globally stable if there is an $\mathbf{x}^{*}\in X$ such that for any $\mathbf{x}^{0}\in X$

[TABLE]

A globally attracting equilibrium $\mathbf{x}^{*}$ in a network is presumably a state at or near which a network can efficiently carry out its function. Whether or not this equilibrium remains stable over time depends on a number of factors including external influences such as changes in the network’s environment, etc. However, not only can outside influences destabilize a network but potentially the network’s own growth can have this effect. As mentioned in the introduction, an important example of this type of destabilization is cancer, which is the abnormal growth of cells that can lead to significant problems in biological networks.

Here we consider how growth via specialization can effect the stability of a network. Specifically, we consider the specialization of a class of dynamical networks $(F,X)$ with components of the form

[TABLE]

where the matrix $A\in\{0,1\}^{n\times n}$ is a matrix of zeros and ones and each $f_{ij}:X_{j}\rightarrow\mathbb{R}$ are $C^{1}(X_{j})$ functions with bounded derivatives for all $i,j\in N$. We refer to the graph $G$ with adjacency matrix $A=\mathcal{A}(G)$ in Equation (3) as the graph of interactions of $(F,X)$.

It is worth noting that we could absorb the matrix $A$ into the functions $f_{ij}$. However, we use this matrix as a means of specializing the dynamical network $(F,X)$ in a way analogous to the method of specialization described in Section 2 for graphs. This is possible as there is a one-to-one relation between a graph $G=(V,E,\omega)$ and its weighted adjacency matrix $\mathcal{A}(G)\in\mathbb{R}^{n\times n}$. Therefore, we can use the notion of a graph specialization to define a matrix specialization.

Definition 7**.**

(Matrix Specialization) Let $A\in\mathbb{R}^{n\times n}$ and $B\subseteq N=\{1,2,\dots,n\}$ be a base. Then the specialization of $A$ over $B$ is the matrix

[TABLE]

where $A=\mathcal{A}(G)$. Additionally, suppose $G=(V,E,\omega)$ and $\mathcal{S}_{B}(G)=(\mathcal{V},\mathcal{E},\mu)$. For $M=\{1,2,\dots,m\}$ let $\tau:M\rightarrow N$ where $\tau(i)=j$ if $\nu_{i}\in\mathcal{V}$ is a copy of $v_{j}\in V$. We refer to the function $\tau$ as the origination function of this specialization.

Note that we are slightly abusing notation in Definition 7 by letting the base $B$ be both a subset of $N=\{1,2,\dots,n\}$ and a subset of $V=\{v_{1},v_{2},\dots,v_{n}\}$. The idea is that $B\subseteq N$ is a set of indices over which the matrix $A$ is specialized, which in turn is the set that indexes the vertices $B\subseteq V$ over which $G=(V,E,\omega)$ is specialized. Roughly speaking, to specialize the matrix $A\in\mathbb{R}^{n\times n}$ we specialize the associated graph $G$ with adjacency matrix $A$. The adjacency matrix of the resulting specialized graph is the specialization of the matrix $A$. This allows us to specialize dynamical networks as follows.

Definition 8**.**

(Specializations of Dynamical Networks) Suppose $(F,X)$ is a dynamical network given by Equation (3). If $B\subseteq\{1,2,\dots,n\}$ then the specialization of $(F,X)$ over the base $B$ is the dynamical network $(G,Y)$ with components

[TABLE]

where $\underline{A}=\mathcal{S}_{B}(A)$, $Y=\bigoplus_{j=1}^{m}Y_{j}$ with $Y_{j}=X_{\tau(j)}$, $y_{j}=x_{\tau(j)}$, and $\tau:M\rightarrow N$ is the origination function of the specialization.

To give an example of a well-studied class of dynamical networks that can be specialized according to Definition 8 consider the class of dynamical networks known as discrete-time recurrent neural networks. The stability of such systems has been the focus of a large number of studies [12, 11, 14, 36], especially time-delayed versions of these systems [25], both of which have the form given in Equation (3).

Definition 9**.**

(Discrete-Time Recurrent Neural Network) A discrete-time recurrent neural network $(R,X)$ is a dynamical network of the form

[TABLE]

where the component $R_{i}$ describes the dynamics of the $i^{th}$ neuron in which each $|a_{i}|<1$ are the feedback coefficients, the matrix $W\in\mathbb{R}^{n\times n}$ with $W_{ii}=0$ is the connection weight matrix, and the constants $c_{i}$ are the exogenous inputs to the network.

In the general theory of discrete-time recurrent neural networks (DRNN) the functions $g_{j}:\mathbb{R}\rightarrow\mathbb{R}$ are typically assumed to be differentiable, monotonically increasing, and bounded. Here, for the sake of illustration, we make the additional assumption that each $g_{j}$ has a bounded derivative.

In the following example we consider the specialization of a DRNN and the dynamic consequences of this specialization.

Example 4.7**.**

(Destabilization via Specialization) Consider the dynamical network $(R,\mathbb{R}^{3})$ given by

[TABLE]

which is the DRNN in which

[TABLE]

where $a_{i}=\frac{9}{10}$ for $i=1,2,3$; $c_{1}=\frac{7}{4}$, $c_{2}=\frac{5}{4}$, $c_{3}=\frac{2}{4}$; with the connection weight matrix

[TABLE]

and $A=W+\frac{9}{10}I$. We choose the function $g_{i}(x)=\tanh(x)$ as this is a standard activation function used to model neural interactions in network science. The network’s graph of interactions $G_{1}$ is shown in Figure 5 (bottom left). As is shown in Figure 5 (top left) the dynamical network $(R,\mathbb{R}^{3})$ is stable with globally attracting fixed point $\mathbf{x}^{*}=(-2.49,2.65,5.05)$.

If the network $(R,\mathbb{R}^{3})$ is specialized over the base $B=\{1,2\}$ the result is the network $(S,\mathbb{R}^{4})$ given by

[TABLE]

The network’s graph of interaction $G_{2}=\mathcal{S}_{B}(G_{1})$ is shown in Figure 5 (top center) where the vertex colors indicate from which network element the elements of $G_{2}$ were specialized. Although the dynamics of the original network $(R,\mathbb{R}^{3})$ is stable, the dynamics of its specialization $(S,\mathbb{R}^{4})$ oscillates periodically as can be seen in Figure 5 (bottom center). That is, specialization does not, at least in general, preserve stability.

The network can again be specialized over the base $C=\{1,2,3\}$ which results in the network of seven elements $(T,\mathbb{R}^{7})$ given by

[TABLE]

The network’s graph of interaction $G_{3}=\mathcal{S}_{C}(G_{2})$ is shown in Figure 5 (bottom right) where, as before, the vertex colors indicate from which network element the elements of $G_{3}$ were specialized. Here specialization has again altered the dynamics of the network as $(T,\mathbb{R}^{7})$ has unstable but now synchronizing dynamics. Specifically, elements $z_{4}$ and $z_{6}$ and also elements $z_{5}$ and $z_{7}$ synchronize irrespective of the network’s initial condition (see Figure 5 top right).

In previous studies of dynamical networks the goal has been to determine under what condition(s) a given network has stable dynamics (see for instance the references in [25]). Here we consider a different but related question which is, under what condition(s) does a dynamical network with an evolving structure of interactions maintain its stability as it evolves.

As a partial answer to this general question, we use the following notion of a stability matrix, which allows us to study the change or lack of change in a network’s stability after it has been specialized.

Definition 10**.**

(Stability Matrix) For the dynamical network $(F,X)$ suppose there exist finite constants

[TABLE]

Then we call the matrix $\Lambda\in\mathbb{R}^{n\times n}$ the stability matrix of $(F,X)$.

The stability matrix $\Lambda$ can be thought of as a global linearization of the typically nonlinear dynamical network $(F,X)$. The following result states that if the eigenvalues of the matrix $\Lambda$ lie within the unit circle then the dynamical network $(F,X)$ is stable, the proof of which can be found in [7].

Theorem 3**.**

(Network Stability) Suppose $\Lambda$ is the stability matrix of the dynamical network $(F,X)$. If $\rho(\Lambda)<1$ then the dynamical network $(F,X)$ is stable.

As it will be helpful in what follows, if $\Lambda$ is the stability matrix of the dynamical network $(F,X)$ we let $\rho(F)=\rho(\Lambda)$ be the spectral radius of the network.

A key feature of the stability described in Theorem 3 is that it is not the standard notion of stability. In [8] it is shown that if $\rho(F)<1$ then the dynamical network $(F,X)$ is not only stable but remains stable even if time-delays are introduced into the network’s interactions (see [8]). Since the addition of time-delays can have a destabilizing effect on a network, the type of stability considered in Theorem 3 is a stronger version of the standard notion of stability. To distinguish between these two types of stability, the stability described in Theorem 3 is given the following name (see [8] for more details).

Definition 11**.**

(Intrinsic Stability) For the dynamical network $(F,X)$, if the spectral radius $\rho(F)<1$ then we say that this network is intrinsically stable.

In the following example we consider the specialization of an intrinsically stable dynamical network.

Example 4.8**.**

(Specialization of an Intrinsically Stable Network) Consider the dynamical network $(\tilde{R},\mathbb{R}^{3})$, which has the same topology as the network $(R,\mathbb{R}^{3})$ in Example 4.8, given by

[TABLE]

which is the DRNN in which

[TABLE]

where $a_{i}=\frac{3}{10}$ for $i=1,2,3$; $c_{1}=\frac{1}{4}$, $c_{2}=c_{3}=0$; with the connection weight matrix

[TABLE]

and $A=\tilde{W}+\frac{3}{10}I$. The graph of interactions $H_{1}$ corresponding to this dynamical network is shown in Figure 6 (bottom left). The dynamical network $(\tilde{R},\mathbb{R}^{3})$ is stable with globally attracting fixed point $\mathbf{x}^{*}=(.19,.13,.23)$, as can be seen in Figure 5 (top left). In fact, the network is intrinsically stable with spectral radius $\rho(S)=.962<1$.

If we specialize this network over the base $B=\{1,2\}$, similar to what is done in the previous example, the result is the network $(\tilde{S},\mathbb{R}^{4})$ given by

[TABLE]

Here the network’s graph of interaction $H_{2}=\mathcal{S}_{B}(H_{1})$ is shown in Figure 5 (bottom center). The vertex colors again indicate from which network element the resulting elements of $H_{2}$ are specialized. Notably, in contrast to the previous example where specialization destabilized the network’s dynamics, the network $(\tilde{S},\mathbb{R}^{4})$ is once again intrinsically stable with $\rho(\tilde{S})=.962$ (see Figure 5, center bottom).

If the network $(\tilde{S},\mathbb{R}^{4})$ is then specialized over the base $C=\{1,3,4\}$ the result in the specialization $(\tilde{T},\mathbb{R}^{7})$ given by

[TABLE]

The network’s graph of interaction $H_{3}=\mathcal{S}_{C}(H_{2})$ is shown in Figure 5 (bottom right). Again one can show that $\rho(\tilde{T})=.962$ so that

[TABLE]

Hence, $(\tilde{T},\mathbb{R}^{7})$ is intrinsically stable (see Figure 5 right bottom). Interestingly, as in the previous example elements $z_{4}$ and $z_{6}$ and also elements $z_{5}$ and $z_{7}$ synchronize irrespective of the network’s initial condition.

Perhaps surprisingly the dynamical network and each of its specializations in the previous example have the same spectral radius. This is not a coincidence but rather a consequence of the similarity between how graph specializations and specializations of dynamical networks are defined. In the proof of the following result we show that if a dynamical network $(F,X)$ is specialized, the resulting dynamical network $(G,Y)$ will have the same spectral radius, i.e. $\rho(F)=\rho(G)$. For example, each of the networks in Example 4.8 have the same spectral radius

[TABLE]

Consequently, the following holds.

Theorem 4**.**

(Stability of Intrinsically Stable Networks Under Specialization) Suppose the dynamical network $(F,X)$ given by equation (3) is intrinsically stable. Then for any $B\subset\{1,2,\dots,n\}$ the specialization $(G,Y)$ of $(F,X)$ over $B$ is intrinsically stable.

Proof.

Suppose the dynamical network $(F,X)$ with stability matrix $\Lambda$ is intrinsically stable so that, in particular, $\rho(\Lambda)<1$. From Equation (3) it follows that

[TABLE]

since $A_{ij}\in\{0,1\}$. If $(G,Y)$ is the specialized version of $(F,X)$ over $S$ with stability matrix $\underline{\Lambda}$ then similarly

[TABLE]

since $\underline{A}=\mathcal{S}_{B}(A)\in\{0,1\}$ where $\tau$ is the origination function of this specialization.

Specializing the stability matrix $\Lambda$ of $(F,X)$ over $B$ results in the matrix

[TABLE]

since $y_{j}=x_{\tau(j)}$, $Y_{j}=X_{\tau(j)}$. Hence, $\mathcal{S}_{B}(\Lambda)=\underline{\Lambda}$ and Corollary 3.3 together with Definition 7 implies that $\rho(\underline{\Lambda})=\rho(\Lambda)<1$ since $\Lambda$ is a nonnegative matrix. Thus, $(G,Y)$ is intrinsically stable. ∎

Theorem 4 is based on the fact that the spectral radius of a graph is preserved under a graph specialization if the graph has nonnegative weights, which every stabilization matrix does (see Corollary 3.3). The importance of Theorem 4 is that it describes, as far as is known to the authors, the first general mechanism for evolving the structure of a network that preserves the network’s dynamics. Currently, it is unknown whether any real-world network intrinsically stable. That is, it is unknown whether any naturally occurring network uses intrinsic stability to maintain its function as its structure evolves. However, from a design point of view, if a network is made to be intrinsically stable, rather than just stable, it is guaranteed to to remain stable so long as its topology evolves under the method of specialization described in Section 2. Moreover, if $(F,X)$, given by (3), is intrinsically stable then any specialization of this network is also intrinsically stable. Hence, any sequence of specializations of the network will result in an intrinsically stable network (cf. Example 4.8 and 4.8).

Corollary 4.9**.**

Suppose $(F,X)$ is an intrinsically stable dynamical network given by (3). If $\{(F^{i},X^{i})\}_{i=1}^{k}$ is any sequence of specialization of $(F,X)$ then $(F^{k},X^{k})$ is intrinsically stable.

A natural and open question is whether this is the case for other types of dynamics exhibited by networks, i.e. whether stronger versions of dynamical behavior such as multistability, periodicity, and synchronization, etc. are required for a network to maintain its dynamics as it grows.

5 Variations of Network Specialization

There are a number of variants of the specialization method that preserve both spectral and dynamic properties of a network. One of the most natural is the partial specialization of a graph (equivalently dynamical network) in which only a subset of the component branches are specialized. This method is demonstrated in the following example

Example 5.10**.**

Let $G$ and $B$ be the graph and base considered in Example 2.2 and shown in Figure 7 (left). In the specialization $\mathcal{S}_{B}(G)$ there are four component branches $\beta_{1},\beta_{2},\beta_{3},\beta_{4}\in\mathcal{B}_{B}(G)$ used to create this graph (see Figure 2 right). If we instead use the two partial component branches $\rho_{1}=\beta_{1}\cup\beta_{2}$ and $\rho_{2}=\beta_{3}\cup\beta_{4}$ the result is the partial specialization $\mathcal{P}_{B}(G)$ shown in 7 (right).

A partial specialization $\mathcal{P}_{B}(G)$ of a graph $G$ over the base $B$ (equivalently dynamical network) is constructed similar to a standard specialization. First the component branches $\mathcal{B}_{B}(G)$ are created. Then we choose a partition of $\mathcal{B}_{B}(G)$, i.e. a collection $\{B_{i=1}^{m}\}$ such that $\mathcal{B}_{B}(G)$ is the disjoint union of this set. We then let $\rho_{i}=\cup_{\beta\in B_{i}}\beta$ be the $i^{th}$ partial component branch for $i=1,\dots,m$. The partial specialization $\mathcal{P}_{B}(G)$ is constructed by merging, i.e. identifying, each vertex $v_{i}\in B$ in any partial branch $\rho_{j}$ with the same vertex $v_{i}$ in any other branch $\rho_{k}$.

We note that the notation $\mathcal{P}_{B}(G)$ does not uniquely define a specific partial specialization of the graph $G$ as there are typically many ways to choose a partition of the component branches $\mathcal{B}_{B}(G)$. It is also worth mentioning that the standard specialization method is the partial specialization where the partition we choose is the one in which every subset has a single element. Hence, a graph specialization is the largest partial specialization over a given base. Additionally, an analogous version of Theorems 1 and 4 as well as Corollary 3.3 hold when we replace the term “specialization” with “partial specialization” in these results.

Proposition 5.11**.**

(Spectral and Dynamic Consequences of Partial Specialization) Let $G=(V,E,\omega)$, $B\subseteq V$, and let $C_{1},\dots,C_{m}$ be the strongly connected components of $G|_{\bar{B}}$. Then for any partial specialization $\mathcal{P}_{B}(G)$ of $G$ we have

[TABLE]

where $n_{i}$ is the number of copies of $C_{i}$ contained in $\mathcal{P}_{B}(G)$. Also if $G$ has positive edge weights then

[TABLE]

Additionally, if the dynamical network $(F,X)$ given by equation (3) is intrinsically stable, then for any $B\subset\{1,2,\dots,n\}$ the partial specialization $(G,Y)$ of $(F,X)$ over $B$ is intrinsically stable.

A proof of Proposition 5.11 is given in Section 6.

Aside from partial specializations another class of specializations we consider is what we refer to as thinned specializations. A thinned specialization $\mathcal{T}_{B}(G)$ of a graph $G$ over the base $B$ (equivalently dynamical network) is a specialization or partial specialization of $G$ in which we do not include some number of component or partial component branches in the specialized graph.

For example, the specialization of the Wikipedia graph in Figure 1 is an example of a thinned specialization. The reason is that a number of component branches have been omitted in this specialization. Specifically, the component branches that begin with one color on the left and end with another color on the right are missing. The fact that these branches have been removed in this example makes sense in the context of a disambiguated network since the network segregates by topic. This type of specialization has following spectral and dynamic properties.

Proposition 5.12**.**

(Spectral and Dynamic Consequences of Tinned Specialization) Let $G=(V,E,\omega)$ have positive edge weights and $B\subseteq V$. Then for any thinned specialization $\mathcal{T}_{B}(G)$ of $G$ we have

[TABLE]

Additionally, if the dynamical network $(F,X)$ given by equation (3) is intrinsically stable, then for any $B\subset\{1,2,\dots,n\}$ the thinned specialization $(G,Y)$ of $(F,X)$ over $B$ is intrinsically stable.

Proof.

Suppose that $G=(V,E,\omega)$ is a graph and $B\subset V$ a base. By Corollary 3.3 it follows that $\rho(\mathcal{S}_{B}(G))=\rho(G)$. Also, under the assumption that $G$ has positive edge weights, $\mathcal{S}_{B}(G)$ has positive edge weights. Thus, given that $\mathcal{T}_{B}(G)$ is a subgraph of $\mathcal{S}_{B}(G)$ then $\rho(\mathcal{T}_{B}(G))\leq\rho(\mathcal{S}_{B}(G))$ (see [20]). Therefore, $\rho(\mathcal{T}_{B}(G))\leq\rho(G)$. Equivalently, for the matrix $A\in\mathbb{R}^{n\times n}$ the spectral radius $\rho(\mathcal{T}_{B}(A))\leq\rho(A)$.

Using this fact, if the dynamical network $(F,X)$ is intrinsically stable then $\rho(\Lambda)<1$, where $\Lambda$ is the stability matrix of $(F,X)$. If $(H,Z)$ is the specialization of $(F,X)$ over the base $B\subset\{1,2,\dots,n\}$ then following the proof of Theorem 4, $\underline{\Lambda}=\mathcal{S}_{B}(\Lambda)$ is the stability matrix of $(H,Z)$. Similarly, one can show that $\underline{\Lambda}_{\mathcal{T}}=\mathcal{T}_{B}(\Lambda)$ is the stability matrix of the thinned specialization $(G,Y)$. Since

[TABLE]

then $(G,Y)$ is intrinsically stable. ∎

As a consequence of Theorem 4 and Propositions 5.11 and 5.12 any sequence of specializations of a graph, whether these are standard, partial, thinned or a combination of these transformations, results in a graph whose spectral radius is bounded by the spectral radius of the original graph. Similarly, if an intrinsically stable dynamical network is sequentially specialized using any combination of these transformations the result is an intrinsically stable network.

6 Isospectral Graph Reductions and Proofs

In this section we a give a proof of Theorems 1 and 2. The proof of Theorem 2 relies on the theory of isospectral graph reductions, which we give in part here (see [9] for more details). For $M\in\mathbb{R}^{n\times n}$ let $N=\{1,\ldots,n\}$. In what follows, if the sets $T,U\subseteq N$ are proper subsets of $N$, we denote by $M_{TU}$ the $|T|\times|U|$ submatrix of $M$ with rows indexed by $T$ and columns by $U$. The isospectral reduction of a square real valued matrix is defined as follows.

A proof of Theorem 1 is the following.

Proof.

For the graph $G=(V,E,\omega)$, without loss of generality, let $B=\{v_{1},\dots,v_{\ell}\}$ where $V=\{v_{1},\dots,v_{n}\}$. By a slight abuse in notation we let $B$ denote the index set $B=\{1,\dots,\ell\}$ that indexes the vertices in $B$.

For the moment we assume that the graph $G|_{\bar{B}}$ has the single strongly connected component $S_{1}$. The weighted adjacency matrix $\mathcal{A}(G)=M\in\mathbb{R}^{n\times n}$ then has the block form

[TABLE]

where $U\in\mathbb{R}^{\ell\times\ell}$ is the matrix $U=M_{BB}$, which is the weighted adjacency matrix of $G|_{B}$. The matrix $Z\in\mathbb{R}^{(n-\ell)\times(n-\ell)}$ is the matrix $Z=M_{\bar{B}\bar{B}}$, which is the weighted adjacency matrix of $G|_{\bar{B}}=S_{1}$. The matrix $Y=M_{B\bar{B}}\in\mathbb{R}^{\ell\times(n-\ell)}$ is the matrix of edges weights of edges from vertices in $S_{1}$ to vertices in $G|_{B}$ and $W=M_{\bar{B}B}\in\mathbb{R}^{(n-\ell)\times\ell}$ is the matrix of edge weights of edges from vertices in $G|_{B}$ to vertices in $S_{1}$.

The specialization $\mathcal{S}_{B}(G)$ is the graph in which all component branches have the form

[TABLE]

are merged together with the graph $G|_{B}$ (cf. Figure 8). The weighted adjacency matrix $\hat{M}=\mathcal{A}(\mathcal{S}_{B}(G))$ has the block form

[TABLE]

where each $Y_{i}\in\mathbb{R}^{\ell\times(n-\ell)}$, each $W_{j}\in\mathbb{R}^{(n-\ell)\times\ell}$, $\sum_{i=1}^{y}Y_{i}=Y$ and $\sum_{j=1}^{w}W_{j}=W$. Here, $y\geq 0$ is the number of directed edges from $S_{1}$ to $G|_{B}$. The matrix $Y_{i}$ has a single nonzero entry corresponding to exactly one edge from this set of edges. Similarly, $w\geq 0$ is the number of directed edges from $G|_{B}$ to $S_{1}$. The matrix $W_{j}$ has a single nonzero entry corresponding to exactly one edge from this set of edges. Since there are $w\cdot y$ component branches in $\mathcal{S}_{B}(G)$ containing $S_{1}$ then the matrix $\hat{M}\in\mathbb{R}^{(\ell+p(n-\ell))\times(\ell+p(n-\ell))}$ where $p=w\cdot y$.

To prove the result of Theorem 1 we will use the Schur complement formula, which states that a matrix with the block form of $M$ has the determinant

[TABLE]

if $Z$ is invertible. Here we apply this formula to the block matrix $M-\lambda I$, which results in the determinant

[TABLE]

where the matrix $Z-\lambda I$ is invertible as a result of the proof of Theorem 1.2 in [9]. Similarly, applying the Schur complement formula to the matrix $\hat{M}-\lambda I$ we have the determinant

[TABLE]

where the equality

[TABLE]

can be seen by direct multiplication of the block entries in the matrix $\hat{M}$. Hence,

[TABLE]

implying

[TABLE]

so that Theorem 1 holds if $G|_{\bar{B}}$ has a single strongly connected component $S_{1}$.

In order to prove Theorem 1 in full generality, we will first need to describe an alternative process which generates the specialized graph $\mathcal{S}_{B}(G)$, which we refer to as “stepwise specialization.” We start this process by defining $L_{1}$ to be the set of the strongly connected components of $G|_{\bar{B}}$, and label the components of this set as $L_{1}=\{S_{1,1},S_{2,1},\dots,S_{m,1}\}$. Next we set $G_{1}=G$ and randomly choose a strongly connected component $S_{i_{1},1}\in L_{1}$ with the property that specializing G around $S_{i_{1},1}$ generates a non-trivial specialization, i.e. $\mathcal{S}_{\bar{S}_{i,1}}(G_{1})\neq G_{1}$. That is, when $G$, is specialized over $\bar{S}_{i,1}$, the result is a larger graph, meaning that there is more than one directed edge into or out of $S_{i,1}$. We then let $G_{2}=\mathcal{S}_{\bar{S}_{i,1}}(G_{1})$. Because $S_{i_{1},1}$ is a single strongly connected component, we showed in the beginning of this proof that the only new vertices which appear in $G_{2}$ as $G_{1}$ is specialized must be $w\cdot y$ copies of $S_{i_{1},1}$, where $w$ is the number of edges which are directed into $S_{i_{1},1}$ and $y$ is the number of edges which are directed out of $S_{i_{1},1}$. Now we relabel the collection of all copies of $S_{i_{1},1}$ in the specialized graph $G_{2}$ as $\{S_{i_{1},1},S_{i_{1},2},\dots,S_{i_{1},r_{1}}\}$.

Next we define $L_{2}$ to be the collection of all strongly connected components of $G_{2}|_{\bar{B}}$, which includes all the elements of $L_{1}$, plus the newly relabeled collection of copies of $S_{i_{1},1}$. Thus

[TABLE]

We then define $G_{3}$ by selecting a random element $S_{i_{2},j_{2}}$ of $L_{2}$ which does not induce a trivial specialization of $G_{2}$ and set $G_{3}=\mathcal{S}_{\bar{S}_{i_{2},j_{2}}}(G_{2})$. Again we relabel all of the strongly connected component copies of $S_{i_{2},j_{2}}$ which appear in $G_{3}$ (this includes new copies and any copies that previously existed in the graph) and define $L_{3}$ to be the collection of all strongly connected components of $G_{3}|_{\bar{B}}$. We continue this process inductively, at each step defining $G_{k+1}=\mathcal{S}_{\bar{S}_{i_{k},j_{k}}}(G_{k})$ where $S_{i_{k},j_{k}}$ is randomly chosen from $L_{k}$ and has the property that $\mathcal{S}_{\bar{S}_{i_{k},j_{k}}}(G_{k})\neq G_{k}$. Then we relabel all copies of $S_{i_{k},1}$ in $G_{k}$ as $\{S_{i_{k},1},\dots S_{i_{k},p_{k}}\}$ and define $L_{k+1}$ as the strongly connected components of $G_{k+1}$ with all the newly defined labels.

Note that at each step we only specialize over one strongly connected component of $G_{\bar{B}}$. An important observation about this process is that the set of component branches with respect to base $B$ is invariant over each step. This can be seen by recognizing that there is a one-to-one correspondence between the component branches $\mathcal{B}_{B}(G_{k})$ and $\mathcal{B}_{B}(G_{k+1})$. From this fact we can conclude that

[TABLE]

for all $k\geq 1$, since we defined specialization to be the graph built by identifying elements of $B$ in all component branches of the graph.

We will continue the process described above until all strongly connected components of $G_{K}|_{\bar{B}}$ induce a trivial specialization. At this point, the process terminates and we are left with the final graph $G_{K}$. We claim that eventually this process must reach this condition and terminate. To verify this, suppose the process did not terminate. Then the specialized graph in each step would be strictly larger than the graph in the previous step. Thus the number of vertices in $G_{k}$ must diverge to infinity, which we denote as $\lim_{k\rightarrow\infty}|G_{k}|=\infty$. However, we know that by construction $|G_{k}|\leq|\mathcal{S}_{B}(G_{k})|$. Using Equation (13) we know that $|\mathcal{S}_{B}(G_{k})|=|\mathcal{S}_{B}(G)|<\infty$. Therefore, at some point this process must terminate.

When the process does terminate, we claim the final graph $G_{K}=\mathcal{S}_{B}(G)$. To see this, first recall that we defined $\mathcal{S}_{B}(G)$ to be the graph which consists of the component branches of $\mathcal{B}_{B}(G)$ in which all elements of $B$ are individually identified (see Definition 2). Thus, if $S$ is a strongly connected component of $G|_{\bar{B}}$, then each copy of $S$ found in $\mathcal{S}_{B}(G)$ will have exactly one edge pointing into it and exactly one edge pointing out of it by the definition of a component branch. In the final graph $G_{K}$, there are no more strongly connected components for which specializing gives a larger graph. Therefore, each strongly connected component in $L_{K}$ has exactly one edge directing into it and one edge directing out of it. Since this is true for each strongly connected component, then $G_{K}$ must be composed of a collection of component branches where all corresponding vertices in $B$ are identified. We already showed $G_{K}$ has the same component branches with respect to $B$ as $\mathcal{S}_{B}(G)$. Using this fact and because strongly connected components of $G_{K}|_{\bar{B}}$ are each their own component branch, we can conclude that $G_{K}=\mathcal{S}_{B}(G)$.

Hence, we can repeatedly apply the result given in Equation (12) for a single strongly connected component at each step in our “stepwise specialization process.” Thus the spectrum of the specialized graph $G_{k}$ is the spectrum of the graph $G_{k-1}$ from the previous step together with the eigenvalues $\sigma(S_{i_{k}})^{p_{k}}$ where $p_{k}$ is the number of copies of $S_{i_{k}}$ which were added. Therefore the eigenvalues of the final graph will be the eigenvalues of the original graph together with the eigenvalues of all the copies of the strongly connected components we added at each step. That is

[TABLE]

where $n_{i}$ is the number of component branches in $\mathcal{B}_{B}(G)$ containing $S_{i}$, completing the proof.

∎

We note that Proposition 5.11 can be proved analogously to Theorem 2. The only difference being that we do not fully specialize every branch (cf. Figure 8) but only partially specialize them. The key observation to proving this result is that Equation 11 also holds in this case in which $Y_{i}$ and $W_{j}$ may have more than one nonzero entry but $\sum_{i=1}^{y}Y_{i}=Y$ and $\sum_{y=1}^{w}W_{i}=W$. Second, the fact that a network with positive edge weights and its partial specialization have the same spectral radius can be shown as in the proof of Corollary 3.3, which verifies the result.

To prove Theorem 2 we need the notion of an isospectral graph reduction, which is a way of reducing the size of a graph while essentially preserving its set of eigenvalues. Since there is a one-to-one relation between the graphs we consider and the matrices $M\in\mathbb{R}^{n\times n}$, there is also an equivalent theory of isospectral matrix reductions. Both types of reductions will be useful to us.

For the sake of simplicity we begin by defining an isospectral matrix reduction. For these reductions we need to consider matrices with rational function entries. The reason is that, by the Fundamental Theorem of Algebra, a matrix $A\in\mathbb{R}^{n\times n}$ has exactly $n$ eigenvalues including multiplicities. In order to reduce the size of a matrix while at the same time preserving its eigenvalues we need something that carries more information than scalars, which for us are rational functions. The specific reasons for using rational functions can be found in [9], chapter 1.

We let $\mathbb{W}^{n\times n}$ be the set of $n\times n$ matrices whose entries are rational functions $p(\lambda)/q(\lambda)\in\mathbb{W}$, where $p(\lambda)$ and $q(\lambda)\neq 0$ are polynomials with real coefficients in the variable $\lambda$. The eigenvalues of the matrix $M=M(\lambda)\in\mathbb{W}^{n\times n}$ are defined to be solutions of the characteristic equation

[TABLE]

which is an extension of the standard definition of the eigenvalues for a matrix with complex entries.

Definition 12**.**

(Isospectral Matrix Reduction) The isospectral reduction of a matrix $M\in\mathbb{R}^{n\times n}$ over the proper subset $B\subseteq N$ is the matrix

[TABLE]

A proof of Theorem 2 part (i) is based on the following result relating the eigenvectors of the a graph $G$ and its reduction $\mathcal{R}_{B}(G)$.

Theorem 5**.**

(Eigenvectors of Reduced Matrices) Suppose $M\in\mathbb{R}^{n\times n}$ and $B\subseteq N$. If $(\lambda,\mathbf{v})$ is an eigenpair of $M$ and $\lambda\notin\sigma(M_{\bar{B}\bar{B}})$ then $(\lambda,\mathbf{v}_{B})$ is an eigenpair of $\mathcal{R}_{B}(M)$.

Proof.

Suppose $(\lambda,\mathbf{v})$ is an eigenpair of $M$ and $\lambda\notin\sigma(M_{\bar{B}\bar{B}})$. Then without loss in generality we may assume that $\mathbf{v}=(\mathbf{v}_{B}^{T},\mathbf{v}_{\bar{B}}^{T})^{T}$. Since $M\mathbf{v}=\lambda\mathbf{v}$ then

[TABLE]

which yields two equations the second of which implies that

[TABLE]

Solving for $\mathbf{v}_{\bar{B}}$ in this equation yields

[TABLE]

where $M_{\bar{B}\bar{B}}-\lambda I$ is invertible given that $\lambda\notin\sigma(M_{\bar{B}\bar{B}})$.

Note that

[TABLE]

Since $(M-\lambda I)\mathbf{v}=0$ it follows that $(\lambda,\mathbf{v}_{B})$ is an eigenpair of $\mathcal{R}_{B}(M)$.

Moreover, we observe that if $(\lambda,\mathbf{v}_{B})$ is an eigenpair of $\mathcal{R}_{B}(M)$ then by reversing this argument, $\vskip 12.0pt plus 4.0pt minus 4.0pt(\lambda,(\mathbf{v}_{B}^{T},\mathbf{v}_{\bar{B}}^{T})^{T}\vskip 12.0pt plus 4.0pt minus 4.0pt)$ is an eigenpair of $M$ where $\mathbf{v}_{\bar{B}}$ is given by (14). ∎

We now give a proof of Theorem 2 part (i).

Proof.

As in the proof of part (i) let $M=\mathcal{A}(G)$ and $\hat{M}=\mathcal{A}(\mathcal{S}_{B}(G))$ where $B$ is a base of $G$. By slightly abusing our notation we let $B$ also denote the subset of integers $B\subset\{1,\dots,N\}$ that indexes the vertices in this base. If $(\lambda,\mathbf{v})$ is an eigenpair of $M$ and $\lambda\notin\sigma(M_{\bar{B}\bar{B}})$ then Theorem 5 implies that $(\lambda,\mathbf{v}_{B})$ is an eigenpair of $\mathcal{R}_{B}(M)$. The claim is that $(\lambda,\mathbf{v}_{B})$ is also an eigenpair of of the reduced matrix $\mathcal{R}_{B}(\hat{M})$.

To see this, first we note that the matrix $M$ and $\hat{M}$ have the form given in (9) and (10), respectively. By Definition 12 the reduced matrices $\mathcal{R}_{B}(M)$ and $\mathcal{R}_{B}(\hat{M})$ are

[TABLE]

Now using equation (11) in the proof of Theorem 1, we have

[TABLE]

implying $\mathcal{R}_{B}(\mathcal{S}_{B}(M))=\mathcal{R}_{B}(M).$ Using this fact and the observation in the last line of the proof of Theorem 5 it follows that $(\lambda,\hat{\mathbf{v}})$ is an eigenpair of $\hat{M}$ where

[TABLE]

Note that $\mathbf{v}_{B}=\hat{\mathbf{v}}_{B}$, which completes the proof. ∎

We now prove parts (ii) and (iii) of Theorem 2. We begin with a proof of part (ii)

Proof.

Let $Z$ be a strongly connected component of $G|_{\overline{B}}$ and let $Z_{i}$ be a copy of $Z$ in $\mathcal{S}_{B}(G)$. Then there exists a component branch $\beta$ corresponding to $Z_{i}$, of the form $\beta=\{v_{j},e_{0},C_{1},...,C_{k},e_{k},Z,e_{k+1}C_{k+1},...,C_{n}\}$ . Then, because specialization isolates each component branch, we can write the adjacency matrix $A$ of $\mathcal{S}_{B}(G)$ in the form,

[TABLE]

Here, $\underline{B}=\mathcal{A}(G|_{B})$, $\underline{Z}_{i}=\mathcal{A}(Z_{i})$ and for $1\leq j\leq k$, $\underline{C}_{j}=\mathcal{A}(C_{j})$. For $0\leq j\leq k+1$, the matrix $Y_{j}$ is a single entry matrix corresponding to the edge $e_{j}\in\beta$. The matrix $X$ is the adjacency matrix for $G$ restricted to nodes that are not in $In(\beta,Z)$, $W$ contains information about edges from nodes represented in $X$ to nodes in $B$.

Suppose $(\lambda,\mathbf{v})$ is an eigenpair of $G$, i.e. $A\mathbf{v}=\lambda\mathbf{v}$, such that $\lambda\notin\sigma(G|_{\bar{B}})$. As $\sigma(G|_{\bar{B}})=\cup_{j=1}^{m}\sigma(C_{i})$ where $C_{1},C_{2},\dots,C_{m}$ are the strongly connected components of $G|_{\bar{B}}$ then $\lambda$ is not an eigenvalue of any strongly connected component of $\beta$.

We may conformally partition $\mathbf{v}$ into $\mathbf{v}=[\mathbf{v}_{B},\mathbf{v}_{C_{1}},\cdots\mathbf{v}_{C_{k}},\mathbf{v}_{{Z_{i}}},\mathbf{v}_{X}]^{T}$ so that entries in each piece correspond with the appropriate sub-matrix of $A$. Then, applying the eigenvector equation produces,

[TABLE]

We solve for $\mathbf{v}_{Z}$ in terms of $\mathbf{v}_{B}$. Multiplying $\mathbf{v}$ by the appropriate block row of $A$ gives $Y_{k}\mathbf{v}_{C_{k}}+\underline{Z}_{i}$, implying that

[TABLE]

[TABLE]

In a similar manner for $2\leq i\leq k$, we may solve for $\mathbf{v}_{C_{i}}$ in terms of $\mathbf{v}_{C_{i-1}}$ producing,

[TABLE]

Combining this with the previous equation yields,

[TABLE]

We solve for $\mathbf{v}_{C_{1}}$ in a similar manner and find that

[TABLE]

Then,

[TABLE]

Since $Y_{0}$,…,$Y_{k}$, $C_{1}$ … $C_{k}$ and $Z_{i}$ are all the appropriate submatrixes of $\mathcal{A}(\,In(\beta,Z_{i})\,)$, by definition of the eigenvector transfer matrix

[TABLE]

Note that each inverse in this equation exists since $\lambda\notin\sigma(Z_{i})$ and $\lambda\notin\sigma(C_{j})$ for $j=1,2,\dots,k$. This completes the proof. ∎

To prove part (iii) of Theorem 2 we use the following. Given a graph $H=(V,E,\omega)$ that is not strongly connected, let $S=\{S_{1},S_{2},...,S_{k}\}$ be the strongly connected components of $H$. If there exist edges $e_{1}\cdots e_{k-1}$ such that, $e_{j}$ is an edge from $S_{j}$ to $S_{j+1}$ for $1\leq j\leq k-1$, we call the ordered set $\alpha$ = {$S_{1}$, $e_{1}$, $S_{2},...,S_{k}$} a partial component branch from $S_{1}$ to $S_{k}$ in $H$. We let $\mathcal{P}(S_{1},S_{k})$ denote the set of all partial component branches from $S_{1}$ to $S_{k}$ in $H$. We let $\mathcal{P}(S_{i},S_{i})$ be the set containing only $\alpha=\{S_{i}\}$.

Definition 13**.**

(Partial Eigenvector Transfer Matrix) Let $H=(V,E,\omega)$ be a graph that is not strongly connected and let $S$ and $T$ be strongly connected components of $H$. For $\alpha=\{S,e_{0},C_{1},e_{1},...C_{m},e_{m}T\}\in\mathcal{P}(S,T)$, let the adjacency matrix of $\alpha$ be

[TABLE]

Where $\underline{S}=\mathcal{A}(S)$, $\underline{T}=\mathcal{A}(T)$ and $\underline{C}_{i}=\mathcal{A}(C_{i})$ for $1\leq i\leq m$. We call the matrix

[TABLE]

the partial eigenvector transfer matrix of $\alpha$, where $\lambda$ is a spectral parameter.

Lemma 1**.**

Suppose G = $(V,E,\omega)$ is not strongly connected with strongly connected components $S_{1}$, $S_{2}$,…,$S_{k}$ and

[TABLE]

where $\underline{S}_{i}=\mathcal{A}(S_{i})$ for $1\leq i\leq k$. If $\lambda\notin\sigma(G)$ then,

[TABLE]

where

[TABLE]

Proof.

Since $A$ is block lower triangular, $(\rho I-A)$ and $(\rho I-A)^{-1}$ are also block lower triangular. Also, since we can write,

[TABLE]

we can write $(\rho I-A)^{-1}$ as,

[TABLE]

where $X_{ii}$ has the same dimensions as $S_{i}$ for $1\leq i\leq k$ and $X_{ij}$ has the same dimensions as $Y_{ij}$ when $1\leq j<i\leq k$. It must then be the case that,

[TABLE]

where each $I_{i}$ is the identity matrix with the same dimensions as $S_{i}$. Let $j\in\{1,2,...,$k$\}$. Then,

[TABLE]

Multiplying the $j$th row of $(\rho I-A)$ by the $j$th column of $(\rho I-A)^{-1}$ produces

[TABLE]

[TABLE]

Since $\mathcal{P}(S_{j},S_{j})$ only contains $\{S_{j}\}$ by definition, it follows that

[TABLE]

We will show by induction that

[TABLE]

when $i>j$. As a base case, consider $X_{(j+1)j}$. By multiplying row $(j+1)$ of $(\rho I-A)$ by the $j$th column of $(\rho I-A)^{-1}$, we obtain the equations

[TABLE]

[TABLE]

Let $m$ be the number of nonzero entries in $Y_{(j+1)j}$. Then we can write

[TABLE]

where each $Y_{(j+1)j}^{(k)}$ has one non-zero entry and that entry is equal to a non-zero entry of $Y_{(j+1)j}$. Thus,

[TABLE]

For each $k$, the block-diagonal matrix

[TABLE]

is an adjacency matrix for $\alpha^{(k)}=\{S_{j},e^{(k)},S_{j+1}\}$ where $e^{(k)}$ is an edge from $S_{j}$ to $S_{j+1}$. Then $\alpha^{(k)}\in\mathcal{P}(S_{j},S_{j+1})$ and

[TABLE]

Clearly, $\cup_{k=1}^{n}\{\alpha^{(k)}\}\subset\mathcal{P}(S_{j},S_{j+1})$. We assert that $\cup_{k=1}^{n}\{\alpha^{(k)}\}=\mathcal{P}(S_{j},S_{j+1})$. To verify this, let $\alpha\in\mathcal{P}(S_{j},S_{j+1})$. Then $\alpha=\{S_{j},e,S_{j+1}\}$ because if $\alpha$ contained any other strongly connected component $S_{l}$, it would imply that a path exists from $S_{j}$ to $S_{l}$ to $S_{j+1}$ and because of the structure of $A$, it must be the case that $l<j$ or $j+1<l$. If an edge existed from $S_{j}$ to $S_{l}$ to $S_{j+1}$ the matrix A would have an entry above the diagonal. This is a contradiction. Thus,

[TABLE]

By the induction hypothesis assume that when $i<n$ we have

[TABLE]

Consider $X_{(j+n)j}$. By multiplying the $j+n$th row of $(\rho I-A)$ by the $j$th column of $(\rho I-A)^{-1}$ we obtain the equations

[TABLE]

[TABLE]

As shown in the base case, the first term can be broken up into a sum of partial eigenvector transfer matrices. Let $m_{0}$ be the number of non-zero entries in $Y_{(j+n)j}.$ If we define $Y_{(j+n)j}^{(k)}$ so that each $Y_{(j+1)j}^{(k)}$ has a single nonzero entry that is equal to a distinct non-zero entry of $Y_{(j+1)j}$ and

[TABLE]

then,

[TABLE]

For each $1\leq k\leq m_{0}$, $(\rho I-S_{j+n})^{-1}Y_{(j+n)j}^{(k)}(\rho I-S_{j})^{-1}$ is the partial eigenvector transfer matrix for a distinct partial component branch $\alpha$ in $\mathcal{P}(S_{j+n},S_{j})$ that does not contain any strongly connected components except $S_{j+n}$ and $S_{j}$. Let $D_{0}$ denote the set of all such branches. By definition of an adjacency matrix, $D_{0}$ contains one branch for each non zero entry in $Y_{(j+n)j}$. Thus,

[TABLE]

We consider the other terms in the sum (3). Let $1\leq i\leq n-1$. By the induction hypothesis,

[TABLE]

Let $m_{i}$ represent the number of nonzero entries in $Y_{(j+n)j+i}$. As before we write $Y_{(j+n)j+i}$ as a sum of $m_{i}$ single entry matrices. $Y_{(j+n)j+i}=\sum_{k=1}^{m_{i}}Y_{(j+n)(j+i)}^{(k)}$. Then,

[TABLE]

It is clear that for each $\alpha\in\mathcal{P}(S_{j},S_{j+i})$ and $k$, the term

[TABLE]

is a partial eigenvector transfer matrix for some incoming branch $\gamma\in\mathcal{P}(S_{j},S_{j}+n)$, because the matrix $Y_{(j+n)(j+i)}^{(k)}$ is non zero if and only if an edge exists from $S_{j+i}$ to $S_{j+n}$. If $\mathcal{P}(S_{j},S_{j+i})$ is non empty, there is a branch from $S_{j}$ to $S_{j+i}$, implying that there must be a branch from $S_{j}$ to $S_{j+n}$ with partial eigenvector transfer matrix $(\rho I-S_{j+n})^{-1}Y_{(j+n)(j+i)}^{(k)}P(\alpha)$.

What we see here is that for a given $i$, the term

[TABLE]

is equal to the sum of all centrality transfer matrices for the branches in $\mathcal{P}(S_{j},S_{j+n})$ that pass through $S_{j+i}$ immediately before reaching $S_{j+n}$. Let $D_{i}$ denote the set of all such branches. Then,

[TABLE]

Putting (15), (16), and (17) together gives,

[TABLE]

Let $D=\cup_{i=0}^{n-1}D_{i}$. Then

[TABLE]

Clearly, $D\subset\mathcal{P}(S_{j},S_{j+n})$. We assert that $D=\mathcal{P}(S_{j},S_{j+n})$. To show this, let $\alpha\in\mathcal{P}(S_{j},S_{j+n})$. If $\alpha$ has only two components, then $\alpha=\{S_{j},e,S_{j+n}\}$ for some edge $e$ and $\alpha\in D_{0}\subset D$ by definition of $D_{0}$. If $\alpha$ has more then two components, then it has a second to last component, $S_{j+i}$ where $1\leq i\leq n-1$. By definition of $D_{i}$, $\alpha\in D_{i}$. Thus,

[TABLE]

This concludes the proof. ∎

We now give a proof of part (iii) of Theorem 2.

Proof.

Since each $Z_{k}$ is a copy of $Z$ in $\mathcal{S}_{B}(G)$, each $Z_{k}$ must correspond to a unique component branch. Let $D=\{\beta_{1},...,\beta_{\ell}\}$ be the set of such branches indexed so that $\beta_{k}$ is the unique branch corresponding to $Z_{k}$ for $1\leq k\leq\ell$. Because each $Z_{k}$ has the same outgoing branch, it follows that $In(\beta_{j},Z)\neq In(\beta_{k},Z)$ when $j\neq k$. Otherwise, there would exist $k\neq j$ such that $\beta_{k}=\beta_{j}$ which contradicts uniqueness of each $\beta_{k}$.

If $In(\beta_{k},Z)=\{v_{i},e_{0},C_{1},e_{1},...,e_{n-1},C_{m},e_{m},Z\}$ define $F_{k}=\{C_{1},C_{2},...C_{m}\}$ to be the set of all strongly connected components of $G|_{\overline{B}}$ in $In(\beta_{k},Z)$. We let $F=\cup_{k=1}^{\ell}F_{k}$. Thus $F$ is the set of all strongly connected components of $G|_{B}(Z)$ that appear before $Z$ in some incoming branch of $D$.

We may order $F=\{C_{1},C_{2},...,C_{n},\}$ such that if $1\leq i<j\leq n$ there are no paths from $C_{j}$ to $C_{i}$. If no such ordering existed, it would imply for some $1\leq i<j\leq n$, paths exist both from $C_{i}$ to $C_{j}$ and from $C_{j}$ back to $C_{i}$. This implies that $C_{i}$ and $C_{j}$ must be part of the same strongly connected component which is a contradiction.

Thus, we can write the adjacency matrix $A=\mathcal{A}(G)$ of $G$ in the form

[TABLE]

where $L$ is of the form,

[TABLE]

and $\underline{B}=\mathcal{A}(G|_{B})$, $\underline{Z}=\mathcal{A}(Z)$, $\underline{C}_{i}=\mathcal{A}(C_{i})$ for $1\leq i\leq m$. Since, $\lambda$ is an eigenvalue of $G$, there is a vector $\mathbf{u}$ such that $A\mathbf{u}=\lambda\mathbf{u}$. We may partition $\mathbf{u}$ into $\mathbf{u}=[\mathbf{u}_{B},\mathbf{u}_{L},\mathbf{u}_{Z},\mathbf{v}_{X}]^{T}$ so that the number of entries in each sub-vector corresponds with the size of the appropriate sub-matrix of $A$

We apply the eigenvector equation to solve for $\mathbf{u}_{Z}$. Given that

[TABLE]

we have

[TABLE]

[TABLE]

Solving for $\mathbf{u}_{L}$ produces,

[TABLE]

Thus,

[TABLE]

We now show that,

[TABLE]

by showing

[TABLE]

First, we consider $(\lambda I-\underline{Z})^{-1}Y_{ZB}$. Since every non-zero entry in $Y_{ZB}$ corresponds to an edge from $G|_{B}$ to $Z$ we let $n_{0}$ equal the number of non-zero entries in $Y_{ZB}$ and write,

[TABLE]

where each $Y_{ZB}^{(r)}$ has exactly one non-zero entry and that entry is equal to a non-zero entry of $Y_{ZB}$. Thus,

[TABLE]

We fix $r\in\{1,...,n_{0}\}$ and consider $(\lambda I-\underline{Z})^{-1}Y_{ZB}^{(r)}$. The matrix $Y_{ZB}^{(r)}$ corresponds to exactly one edge from $G|_{B}$ to $Z$. Therefore, there must exist $\beta_{r}\in D$ such that $In(\beta_{r},Z)=\{v,e,Z\}$ where $v\in B$ and $e$ corresponds with the non-zero entry in $Y_{ZB}^{(r)}$. Hence,

[TABLE]

Furthermore, $\beta_{r}$ must be the only branch in $D$ that contains $e$. If not, there must be another $\beta_{s}\in D$ such that $In(\beta_{s},Z)=\{v,e,Z\}=In(\beta_{r},Z)$. Since $\beta_{r}$ and $\beta_{s}$ are in $D$, they have the same outgoing branch and it must be the case that $\beta_{r}=\beta_{s}$. This contradicts uniqueness of each $\beta$ in $D$.

Thus, each $Y_{ZB}^{(r)}$ corresponds with exactly one $\beta_{r}\in D$. Furthermore, we see that $In(\beta_{r},Z)$ does not contain any strongly connected components except $Z$, because the branch contains an edge directly from a node in $B$ to $Z$. Thus, we may write it’s adjacency matrix as follows:

[TABLE]

Then $(\lambda I-\underline{Z})^{-1}Y_{ZB}^{(r)}$ must the eigenvector transfer matrix of $\beta_{r}$ with respect to $Z$.

Consider the set $D_{0}=\{\beta_{1},...\beta_{n_{0}}\}$ of component branches corresponding to $\{Y_{ZB}^{(1)},...,Y_{ZB}^{(n_{0})}\}$. In this case

[TABLE]

Note that for all $\beta\in D_{0}$, $In(\beta,Z)$ has no strongly connected components except $Z$. We assert that $D_{0}$ is the set of all branches in $D$ that satisfy this property. To show this, note that if $\beta\in D$ and $In(\beta,Z)=\{v,e,Z\}$, then by definition of $\mathcal{A}(G)$, $e$ must correspond to a non-zero entry in $Y_{ZB}$ and therefore $\beta$ corresponds to $Y_{ZB}^{(r)}$ for some $1\leq r\leq n_{0}$. Then $\beta\in D_{0}$ and $D_{0}$ is the set of all $\beta\in D$ where $In(\beta,Z)$ contains no strongly connected components except $Z$.

Next we consider $(\lambda I-\underline{Z})^{-1}Y_{ZL}(\lambda I-L)^{-1}Y_{ZB}$ from equation (18). Once again, we write $Y_{ZL}=\sum_{s=1}^{n_{1}}Y_{ZL}^{(s)}$ and $Y_{LB}=\sum_{t=1}^{n_{2}}Y_{LB}^{(t)}$ as the sum of their non-zero entries.

By definition of an adjacency matrix, it must be the case that the set $\{Y_{ZL}^{(s)}\}_{s=1}^{n_{1}}$ is in a bijective correspondence with the edges from the components in $\{C_{1},...C_{n}\}$ to $Z$ and the set $\{Y_{LB}^{(t)}\}_{t=1}^{n_{2}}$ is in a bijective correspondence with the edges from $G|_{B}$ to components in $\{C_{1},...C_{n}\}$. Thus we may let $\{f_{s}\}_{s=1}^{n_{1}}$ be the set of edges corresponding with $\{Y_{ZL}^{(s)}\}_{s=1}^{n_{1}}$ and $\{g_{s}\}_{t=1}^{n_{1}}$ be the set of edges corresponding with $\{Y_{LB}^{(s)}\}_{t=1}^{n_{1}}$.

We, therefore, have

[TABLE]

Fix, $s\in\{1,...,n_{1}\}$ and $t\in\{1,...n_{2}\}$ and consider, $Y_{ZL}^{(s)}(\lambda I-L)^{-1}Y_{LB}^{(t)}$. By definition $Y_{LB}^{(s)}$ represents an edge from $G|_{B}$ to some $C_{i}\in F$ and $Y_{ZL}^{(s)}$ represents an edge from some $C_{j}\in F$ to $Z$. Since there are no paths from $C_{j}$ to $C_{i}$ when $i>j$, it must be the case that $i\leq j$. This gives us information about the location of the non-zero entries in $Y_{LB}^{(s)}$ and $Y_{ZL}^{(s)}$. In particular, it must be the case that

[TABLE]

For some $Y^{(t)}$ and $Y^{(s)}$ that contain a single non-zero entry. This is because $Y_{LB}^{(t)}$ and $Y_{ZL}^{(s)}$ represent an edge from $G|_{B}$ to a component $C_{i}$ and an edge from a component $C_{j}$ to $Z$ respectively. Thus, we conclude that all entries in $Y_{LB}^{(t)}$ and $Y_{ZL}^{(s)}$ that correspond to edges to or from components besides $C_{i}$ and $C_{j}$ respectively must be zero.

By lemma 1

[TABLE]

where

[TABLE]

Using this fact we simplify the expression $Y_{ZL}^{(s)}(\lambda I-L)^{-1}Y_{LB}^{(t)}$ to

[TABLE]

demonstrating that,

[TABLE]

Thus, for each $s$ and $t$ the term,

[TABLE]

where $j_{s},i_{t}\in\{1,...,n\}$.

We now show for each $\alpha\in\mathcal{P}(C_{i_{t}},C_{j_{s}})$ that

[TABLE]

where $\beta\in\mathcal{B}_{B}(G)$. Let $\alpha\in\ \mathcal{P}(C_{i_{t}},C_{j_{s}})$. Then $\alpha=\{C_{\alpha_{0}},e_{1},C_{\alpha_{1}},e_{2},...C_{\alpha_{N-1}},e_{N}C_{\alpha_{N}}\}$ where $C_{\alpha_{0}}=C_{i_{t}}$, $C_{\alpha_{N}}=C_{j_{s}}$ and for each $1\leq k\leq N$ we have $C_{\alpha_{k}}\in F$. Because $Y^{(s)}$ and $Y^{(t)}$ correspond to unique edges from $C_{j_{s}}$ to $Z$ and from $G|_{B}$ to $C_{i_{t}}$, respectively, there is a unique component branch $\beta\in\mathcal{B}_{B}(G)$ such that

[TABLE]

for some $v,u\in B$ and where $g_{t},f_{s}\in E$ are edges corresponding to $Y^{(t)}$ and $Y^{(s)}$ respectively as defined previously. Thus, both

[TABLE]

[TABLE]

because, $P(\alpha,\lambda)=(\lambda I-\underline{C}_{\alpha_{0}})^{-1}Y_{1}(\lambda I-\underline{C}_{\alpha_{1}})^{-1}Y_{2}...Y_{N}(\lambda I-\underline{C}_{\alpha_{N}})^{-1}$.

We see given $s$, $t$ and $\alpha$ that $(\lambda I-\underline{Z})^{-1}Y^{(s)}P(\alpha,\lambda)Y^{(t)}$ is equal to the eigenvector transfer matrix for some branch $\beta$ that contains $Z$ where the first edge in $\beta$ is $g_{t}$ and the edge before $Z$ is $f_{s}$. Let $D_{st}$ represent the set of all $\beta\in\mathcal{B}_{B}(G)$ where the first edge is $g_{t}$ and the edge leading to $Z$ is $f_{s}$. We have already shown that for each $\alpha\in\mathcal{P}(C_{i_{t}},C_{j_{s}})$, there exists a $\beta\in D_{st}$ such that

[TABLE]

We now show that for each $\beta\in D_{st}$ there exists an $\alpha\in\mathcal{P}(C_{i_{t}},C_{j_{s}})$ such that the above equation is true. Let $\beta\in D_{st}$. Then $\beta=\{v,g_{t},C_{\beta_{0}},e_{1}...e_{M},C_{\beta_{M}},f_{s},Z...u\}$ where for $1\leq k\leq M$, $e_{k}\in E$ and $C_{\beta_{k}}\in F$ when $0\leq k\leq M$. By definition of a component branch, each $e_{k}$ is and edge from $C_{\beta_{k-1}}$ to $C_{\beta_{k}}$ when $1\leq k\leq M$. This implies that there is a partial component branch from $C_{\beta_{1}}$ to $C_{\beta_{M}}$ of the form, $\alpha=\{C_{\beta_{0}},e_{1}...e_{M},C_{\beta_{M}}\}$. Clearly $\alpha\in\mathcal{P}(C_{\beta_{1}},C_{\beta_{M}})$. Hence, the adjacency matrix for $\alpha$ is of the form,

[TABLE]

and $P(\alpha,\lambda)=(\lambda I-\underline{C}_{\beta_{M}})^{-1}Y_{M}...Y_{1}(\lambda I-\underline{C}_{\beta_{0}})^{-1}$. Since the components and edges in $\alpha$ are in $\beta$, the adjacency matrix of $In(\beta,Z)$ can be written as

[TABLE]

Thus, there exists an $\alpha$ such that $T(\beta,Z,\lambda)=(\lambda I-Z)^{-1}Y^{(s)}P(\alpha,\lambda)Y^{(t)}$. Thus, the sets $\mathcal{P}(C_{i_{t}},C_{j_{s}}$ are in a bijective correspondence.

We know that for each $\alpha\in\mathcal{P}(C_{i_{t}},C_{j_{s}})$ there is a $\beta\in D_{st}$ such that

[TABLE]

Using this fact, we may substitute each $(\lambda I-\underline{Z})^{-1}Y^{(s)}P(\alpha,\lambda)Y^{(t)}$ in the sum

[TABLE]

for the corresponding $T(\beta,Z,\lambda)$, ($\beta\in D_{st}$ producing

[TABLE]

We know that every $\beta\in D_{st}$ is accounted for in the sum because of the correspondence demonstrated previously.

We have now shown that

[TABLE]

Since

[TABLE]

by equation (19) we have

[TABLE]

Finally we show that

[TABLE]

By definition, $D_{0}\subset D$ and $D_{st}\subset D$ for all $1\leq s\leq n1,1\leq t\leq n_{2}$. Then $D_{0}\cup\left(\bigcup_{s=1}^{n_{1}}\bigcup_{t=1}^{n_{2}}D_{st}\right)\subset D$. Let $\beta\in D$. Then $In(\beta,Z)$ either contains a strongly connected component besides Z, or it does not. If it does not, $\beta\in D_{0}$ as shown previously. If it does, then the first edge in $\beta$ must connect a vertex in $G|_{B}$ to a vertex in some component $C_{i}\in F$ and therefore correspond to $f_{t}$ for some $1\leq t\leq n_{2}$. Additionally the edge that appears prior to $Z$ must correspond to $g_{s}$ for some $1\leq s\leq n_{1}$. Thus $\beta\in D_{0}\cup\left(\bigcup_{s=1}^{n_{1}}\bigcup_{t=1}^{n_{2}}D_{st}\right)$ and $D\subset D_{0}\cup\left(\bigcup_{s=1}^{n_{1}}\bigcup_{t=1}^{n_{2}}D_{st}\right)$. This produces

[TABLE]

By part (ii) of Theorem 2 we have that $T(\beta_{k},Z,\lambda)\mathbf{u}_{B}=\mathbf{v}_{Z_{k}}$ and

[TABLE]

This completes the proof of part (iii). ∎

7 Concluding Remarks

In this paper we consider the interplay of the dynamics on a network and the dynamics of a network, where the dynamics of the network is modeled by the process of network specialization and the type of dynamics on the network we consider is global stability. What allows us to bridge these two types of dynamics is knowing how specialization effects the spectral properties of the network. In this way we are able to rigourously study the effect the evolving structure of the network has on the dynamics of the network elements.

In a following paper we plan to use this model to study the full interplay of these two types of network dynamics. Specifically, how the dynamics on the network effects the structural evolution of the network and vice-versa and what the eventual structure and dynamics of such networks are.

It is worth mentioning that the extent to which the specialization model models the growth of real-world networks is still unknown. The reason is that, as a network (graph) can be specialized over any subset of its elements (vertices) these are many ways to grow a network especially considering the new variants of network specialization introduced in this paper. An open question is, for specific real-world network, whether a rule can be devised for selecting a base or sequence of bases that grows a small network into a network that has some of the “finer details” of the real network, e.g. similar features, specific statistics, etc.

Another aspect of the paper that is worth addressing here is the notion of intrinsic stability. An interesting feature of intrinsic stability, not shared by the standard version of stability, is its resilience to change in network structure. Not only is intrinsic stability maintained under specialization but it is also maintained when time delays are introduced into the network’s interactions (see [8] and [32]). As time-delays lengthen the paths and cycles between network elements in the network’s graph of interactions these time delays have an effect on the network’s topology, which are distinct from the topological changes caused by specialization.

An open question is what other types of intrinsic dynamics exist, which analogous to intrinsic stability are maintained under structural changes to the network. Such stronger forms of the standard forms of network dynamics would potentially suggest how real networks maintain their function even as their structure evolves as intrinsic stability does for stable networks.

8 Acknowledgement

The work of L. A. Bunimovich was partially supported by the NSF grant DMS-1600568. The work of B. Z. Webb is partially supported by was partially supported by the DOD grant HDTRA1-15-0049.

References