Network-ensemble comparisons with stochastic rewiring and von Neumann entropy

TL;DR

This paper introduces a framework for comparing networks to random ensembles using stochastic rewiring and von Neumann entropy, providing a computationally efficient method to assess network typicality.

Contribution

It develops a novel approach combining stochastic rewiring and von Neumann entropy to quantify network similarity to random ensembles without extensive simulations.

Findings

Rewiring processes converge to Erdos-Renyi and configuration models.

Von Neumann entropy effectively measures information content changes.

Estimated rewiring steps needed for network to resemble ensemble.

Abstract

Assessing whether a given network is typical or atypical for a random-network ensemble (i.e., network-ensemble comparison) has widespread applications ranging from null-model selection and hypothesis testing to clustering and classifying networks. We develop a framework for network-ensemble comparison by subjecting the network to stochastic rewiring. We study two rewiring processes, uniform and degree-preserved rewiring, which yield random-network ensembles that converge to the Erdos-Renyi and configuration-model ensembles, respectively. We study convergence through von Neumann entropy (VNE), a network summary statistic measuring information content based on the spectra of a Laplacian matrix, and develop a perturbation analysis for the expected effect of rewiring on VNE. Our analysis yields an estimate for how many rewires are required for a given network to resemble a typical network from an ensemble, offering a computationally efficient quantity for network-ensemble comparison that does not require simulation of the corresponding rewiring process.

Figures

Tables

Table 1. Table 1 : Empirical networks studied in Fig.  5 .

Network	$N$	$M$	Reference
dolphin social network	62	159	[46]
Les Misérables characters	77	254	[41]
word adjacency in David Copperfield	112	425	[56]
jazz collaborations	198	2742	[33]
C. elegans neuronal network	297	2148	[82]
C. elegans metabolic network	453	2025	[25]
world airport network	500	2980	[21]
Caltech Facebook network	762	16651	[81]
university email messages	1133	5451	[36]
U. S. power grid	4941	6594	[82]

Equations

$$A_{ij}=\{\begin{array}{ll}1,& (i,j)\in \mathcal{E}\\ 0,& \text{mbox}otherwise.\end{array}$$

$$L=D-A.$$

$$h(G)=-\mathrm{Tr}(\mathcal{L}{\log }_2\mathcal{L}).$$

$$h(G)=-\sum _{i=1}^N\frac{{\lambda }_i}{2M}{\log }_2\left(\frac{{\lambda }_i}{2M}\right)$$

$${\hat{\pi }}_s^{(t+1)}=\sum _r{\hat{\pi }}_r^{(t)}{\hat{P}}_{rs},$$

$$\lim_{t\to\infty}\hat{{\pi}}_{s}^{(t)}={\color[rgb]{0,0,0}\hat{{S}}}_{N,\mathbf{d}}^{-1}.$$

$${\pi }_s^{(t+1)}=\sum _r{\pi }_r^{(t)}{P}_{rs},$$

$$\underset{t\to \infty }{\lim }{\pi }_s^{(t)}=S_{N,M}^{-1}.$$

$${\mathbf{G}}^{(t)}\to \mathbf{G}\iff {\pi }^{(t)}\to \pi .$$

$$\underset{t\to \infty }{\lim }{\mathbf{G}}^{(t)}={\mathbf{G}}_{N,M}.$$

$$\lim_{t\to\infty}\hat{\mathbf{G}}^{(t)}={\color[rgb]{0,0,0}\hat{{\mathbf{G}}}}_{N,\mathbf{d}}.$$

$$P^{(t)}(f)$$

$${\hat{P}}^{(t)}(f)$$

$$P^{(t)}(f)$$

$${\hat{P}}^{(t)}(f)$$

$$P^{(t)}(h)$$

$${\hat{P}}^{(t)}(h)$$

$${\lambda }_i(ϵ)$$

$${\mathbf{v}}^{(i)}(ϵ)$$

$${\lambda }_i^{\prime }(0)$$

$${{\mathbf{v}}^{(i)}}^{\prime }(0)$$

$$h(\epsilon)=h(0)+\epsilon h^{\prime}(0)+\mathcal{O}(\epsilon^{2}){\color[rgb]{0,0,0},}$$

$$h^{\prime }(0)=-\frac{1}{2M}\sum _i({\mathbf{v}}^{(i)}{)}^T\Delta L{\mathbf{v}}^{(i)}\left({\log }_2\left(\frac{{\lambda }_i}{2M}\right)+\frac{1}{\ln (2)}\right).$$

$$\displaystyle\Delta{L}_{ij}^{(pq)}=\left\{\begin{array}[]{rl}\pm 1,&{(i,j)\in\{(p,p),(q,q)\}}\\ \mp 1,&{(i,j)\in\{(p,q),(q,p)\}}\\ 0,&\text{otherwise},\end{array}\right.$$

$$h^{\prime }(0)=-\frac{1}{2M}\sum _{i=1}^N\pm ({\mathbf{v}}_p^{(i)}-{\mathbf{v}}_q^{(i)}{)}^2\left({\log }_2\left(\frac{{\lambda }_i}{2M}\right)+\frac{1}{\ln (2)}\right),$$

$$\Delta L_{ij}^{({\mathcal{E}}^{+},{\mathcal{E}}^{-})}=\sum _{(p,q)\in {\mathcal{E}}^{+}}\Delta L_{ij}^{(pq)}-\sum _{(p,q)\in {\mathcal{E}}^{-}}\Delta L_{ij}^{(pq)},$$

$$h^{\prime }(0)$$

$$\mathbf{E}[\Delta L_{ij}]=\{\begin{array}{ll}\frac{N-1-d_i}{\frac{N(N-1)}{2}-M+1}-\frac{d_i}{M}& \text{mbox}ifi=j\\ \frac{A_{ij}}{M}-\frac{1-A_{ij}}{\frac{N(N-1)}{2}-M+1}& \text{mbox}ifi\ne j\end{array}$$

$$\mathbf{E}[{\lambda }_n^{\prime }(0)]$$

$$\mathbf{E}[{{\mathbf{v}}^{(n)}}^{\prime }(0)]$$

$$\mathbf{E}[h^{\prime }(0)]$$

$${\overline{h}}^{(N,M)}={\int }_0^{\infty }h{P}^{(N,M)}(h^{\prime })d{h}^{\prime }.$$

$${\color[rgb]{0,0,0}H^{(N,M)}}{(\alpha)}=H~{}\text{ such that }~{}\alpha=\int_{0}^{H}P^{(N,M)}(h^{\prime})dh^{\prime},$$

$$\tau_{\alpha}=\mathbf{E}\left[\min_{t}~{}\{t:h_{t}\in[{\color[rgb]{0,0,0}H^{(N,M)}}(\alpha),{\color[rgb]{0,0,0}H^{(N,M)}}(1-\alpha)]\}\right].$$

Full text

{AMS}

94A17, 05C82, 62M02, 60J10, 60Gxx, 28D20

1 Introduction

Numerous social, biological, technological and information systems are naturally manifest as networks [57], and common questions about networks are cast explicitly or relate implicitly to network models and their corresponding network ensembles—a set of networks combined with a sampling probability distribution. The Erdős-Rényi (ER) and configuration model random-network ensembles, for example, have provided cornerstones for the development of graph theory [26, 14, 17] and are widely used as null models for network-data analytics including community detection [53, 58] and significance testing of subgraph motifs [51]. Moreover, many applications call for the simultaneous study of a set of empirical networks, encoded as layers in a multilayer (e.g., multiplex) network [12, 40], where it can be beneficial to study them as independent samples from an ensemble [75, 79, 77].

We pursue here two classes of questions related to network ensembles: network-network comparison and network-ensemble comparison. Network-network comparison aims to identify a similarity measure between networks, for instance as a means for clustering and classifying networks [71, 1, 29, 52, 74, 10, 60, 43, 47, 39, 24, 23, 59, 16]. Closely-related questions of network-ensemble comparison aim to assess whether a given network is typical or atypical for an ensemble (or to quantify how typical). Such comparison is useful for null-model selection and hypothesis testing [7, 16, 42, 51]. Understanding if a given network is typical for an ensemble is particularly important for modeling dynamics on networks (e.g., epidemic spreading [62], social contagions [31], synchronization [68, 72], neuronal excitation [44], percolation theory [18, 78], and so on). Specifically, the accuracies of mean field theories or other model system reductions to describe the expected dynamics for random-network ensembles are related to whether a network is typical or atypical for an ensemble[49, 32, 78, 67].

We study network-network and network-ensemble comparisons through von Neumann entropy (VNE), a summary statistic that measures a network’s information content based on the spectra of its associated Laplacian matrix [15, 61]. VNE-based comparison is closely related to the family of network-network comparisons known as spectral comparisons [30], which relate networks by some function of the eigenvalues of matrices associated with the networks (e.g., adjacency, normalized Laplacian, and unnormalized Laplacian). Our focus on VNE is motivated by recent research [23, 24] that used VNE to hierarchically cluster layers in multilayer networks. We stress, however, that the mathematical techniques that we develop here can be generalized to other summary statistics of networks.

Our main goal is to study VNE for networks undergoing stochastic rewiring. We study two rewiring processes—uniform and degree-preserved rewiring—that yield random-network ensembles that converge in distribution to the ER and configuration-model ensembles, respectively. This convergence follows from studying rewiring as a degree-regular Markov chain in which states represent networks and transitions represent stochastic rewiring. Indeed, stochastic rewiring is an established approach for Markov chain Monte Carlo (MCMC) algorithms for sampling configuration ensembles [28, 11, 50]. Because stochastic rewiring is an important generative model for time-varying networks [37], our theory also provides insight about the VNE of time-varying networks. We conduct a perturbation analysis for the change in VNE incurred by rewiring a small number of edges. We prove that the distribution of network summary statistics (e.g., VNE) for an ensemble of networks obtained by rewiring $t$ edges converges as $t\to\infty$ to the appropriate distribution for the associated random-network ensemble. Combining these two results, we obtain an exponential extrapolation $B_{\alpha}$ that predicts how many rewires are necessary to modify an empirical network so that it resembles a typical network from an ensemble. Importantly, the calculation of $B_{\alpha}$ does not require the full simulation of the stochastic rewiring process, nor the calculation of VNE for rewired networks, and is therefore a computationally efficient quantity for evaluating network-ensemble comparisons.

The remainder of this paper is organized as follows. In Sec. 2, we provide background information. In Sec 3, we present our main mathematical results regarding the VNE of networks undergoing stochastic rewiring. In Sec. 4, we present numerical experiments and introduce the quantity $B_{\alpha}$ for network-ensemble comparisons. We provide a discussion in Sec. 5.

2 Background Information

We now introduce our mathematical notation and provide background information about the Laplacian matrix (Sec. 2.1), VNE (Sec. 2.2), random-network ensembles (Sec. 2.3), and Markov-chain theory for stochastic rewiring (Sec. 2.4).

2.1 Laplacian Matrix of a Network

Let ${G}(\mathcal{E},\mathcal{V})$ denote a network with set $\mathcal{V}=\{1,\dots,N\}$ containing $N=|\mathcal{V}|$ nodes and set $\mathcal{E}\in\mathcal{V}\times\mathcal{V}$ containing $M=|\mathcal{E}|$ edges. We assume the network is simple, unweighted, undirected, and that there are no self-edges. The network can be equivalently defined by a symmetric adjacency matrix

[TABLE]

Note that $\sum_{ij}A_{ij}=2M$ since each of the $M$ edges gives rise to two nonzero entries in $A$. We define the degree matrix to be $D=\mbox{diag}[d_{1},\dots,d_{N}]$, where $d_{i}=\sum_{j}A_{ij}$ is the degree for each node $i$. The unnormalized Laplacian matrix is given by

[TABLE]

The matrix $L$, also known as the combinatorial Laplacian, is important in numerous applications including graph partitioning [27], spanning tree analysis [48], synchronization of nonlinear dynamical systems [63, 73, 80], diffusion of random walks [14, 45], manifold learning [9, 19], and harmonic analysis [20]. In Sec. 3.4, we analyze the expected effect of rewiring on VNE, which requires us to first study the expected effect on $L$. We expect our mathematical results to also potentially benefit these other diverse applications that rely on $L$.

2.2 von Neumann Entropy (VNE)

VNE was introduced by John von Neumann as a measure for quantum information [55] and can quantify, for example, the departure of a quantum-mechanical system from its pure state. Recently, this formalism has been generalized to study information content in networks.

Definition 2.1** (von Neumann Entropy of a Network [15]).**

Let ${G}(\mathcal{E},\mathcal{V})$ denote a network, $L$ denote its associated unnormalized Laplacian defined by Eq. (2), and $\mathcal{L}=L/2M$, where $M$ is the number of undirected edges. The VNE of {\color[rgb]{0,0,0}G}(\mathcal{E},\mathcal{V}) is given by

[TABLE]

Remark 2.2**.**

Since $\mathcal{L}$ is positive semi-definite and $\operatorname{Tr}(\mathcal{L})=\sum_{i}d_{i}/2M=1$, $h$ can be written in terms of the set $\{\lambda_{1},\lambda_{2},\dots,\lambda_{N}\}$ of eigenvalues of $L$ as

[TABLE]

*[By convention we define $0\log_{2}(0)=0$]. Note also that because $\operatorname{Tr}(\mathcal{L})=1$, the variables $\lambda_{i}/2M$ may be interpreted as probabilities. It’s worth noting, however, that although $\mathcal{L}$ shares the mathematical properties of a density matrix (i.e., it’s positive semidefinite and $\operatorname{Tr}(\mathcal{L})=1$), it does not have the same physical meaning as a density matrix in quantum mechanics. *

Remark 2.3**.**

*Recently, Ref. [23] introduced an alternative notion of VNE for networks using density matrix $\mathcal{L}\varpropto e^{-\beta L}$ for $\beta>0$. They showed this version satisfies the subadditivity property, which can be preferable for some applications, but we will not consider this version further in the present work. *

VNE quantifies the information content of a network through the eigenvalues of its associated Laplacian matrix, which are well known to reflect network topology [9, 17, 64, 80]. In particular, previous research studying a random-network ensemble found VNE to be larger for degree-regular networks and smaller for networks with irregular structures such as long paths and nontrivial symmetries [61]. VNE has been receiving growing attention for its utility for network-network comparison and has been used recently to hierarchically cluster layers in multilayer networks [23, 24].

As motivation, we present a numerical experiment illustrating the ability of VNE to distinguish between typical and atypical networks in the Erdős-Rényi $\mathbf{G}_{N,M}$ ensemble of simple random networks (see Definition 2.4 in Sec. 2.3) with $N=25$ and $M=50$. We studied the probability distribution of VNE across the ensemble, $P^{(N,M)}(h)$, which we estimated by sampling $10^{4}$ networks from the ensemble. In Fig. 1(a), we plot the observed distribution $P^{(N,M)}(h)$. The vertical dashed line indicates the empirical mean and solid lines indicate the 5% and 95% quantiles. In Fig. 1(b), we provide a scatter plot that indicates for each of these networks the maximum degree, minimum degree, and degree variance versus VNE. Note that degree heterogeneity negatively correlates with VNE, illustrating that networks with small (large) VNE are more (less) degree irregular. In Fig. 1(c), we provide visualizations for several networks sampled from $\mathbf{G}_{N,M}$, which are arranged so that their VNEs increase from left to right.

The main takeaway from this experiment is that larger VNE can be interpreted to indicate decreased ‘irregularity’ in a network. By studying the distribution of VNE for a random-network ensemble, one can differentiate between networks that are typical (i.e., their VNEs are typical and they contain a typical amount of irregular structure) and those that are atypical (i.e., their VNEs are in the tails of the distribution $P^{(N,M)}(h)$, with either more or less irregularity than is typical). It is worth noting that although this experiment focuses on degree heterogeneity, previous research has established a complicated connection between Laplacian spectra (and hence VNE) and other sources for structural irregularity including subgraph motifs [65, 66], communities [27, 64], manifolds [9, 20], trees/loops [54, 69, 70, 80], and so on. See [3, 2] for connections between VNE and other information-theoretic measures of networks.

2.3 Random-Network Ensembles

We consider two random-network ensembles that have each received considerable attention: the ER and configuration model ensembles.

Definition 2.4** (Erdős-Rényi Ensemble $\mathbf{G}_{N,M}$ of Simple Networks [26]).**

*Let $\mathcal{G}_{N,M}=\{{G}_{s}\}$ denote the set of networks $G_{s}$ with $N$ nodes and $M$ undirected edges, disallowing repeat edges and self-edges. Note that $S_{N,M}=|\{\mathcal{G}_{N,M}\}|=\binom{N(N-1)/2}{M}$. Let ${\bf\pi}$ denote a uniform distribution on $\mathcal{G}_{N,M}$ so that $\pi_{s}=1/S_{N,M}$. The ER random-network ensemble is defined by the pair $\mathbf{G}_{N,M}=(\mathcal{G}_{N,M},{\bf\pi})$. *

The most common approach to sample networks from $\mathbf{G}_{N,M}$ involves enumerating the potential edges $\{1,2,\dots,\binom{N}{2}\}$ and choosing $M$ of them uniformly at random. Whenever considering ER models, it is typically worth noting that there exists another ER model that is closely related, $\mathbf{G}_{N,p}$, in which each pair of nodes is connected by an edge independently with probability $p\in[0,1]$. $\mathbf{G}_{N,M}$ and $\mathbf{G}_{N,p}$ are referred to as the microcanonical and canonical ER models, respectively. They have greatly benefited theory development in network science and graph theory and are arguably the simplest random-network ensembles. Real-world networks, however, are well known to contain a variety of structures not well represented by the ER ensembles. In particular, the probability distribution of node degrees is binomial for the ER models; however, the degree distributions of empirical networks are often much more heavy-tailed (networks with power-law distributions [6] being just one such class). For this reason, there is widespread interest in configuration model random-network ensembles that allow a priori specification of the degree sequence $\mathbf{d}=\{d_{i}\}$.

Definition 2.5** (Configuration Ensemble {\color[rgb]{0,0,0}\hat{\mathbf{G}}_{N,\mathbf{d}}} of Simple Networks [8]).**

*Let {\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}=\{\hat{G}_{s}\} denote the set of networks $\hat{G}_{s}$ with $N$ nodes and degree sequence $\mathbf{d}$, disallowing repeat edges and self-edges. Note that $M=\frac{1}{2}\sum_{i}d_{i}$. Let $\hat{\pi}$ denote a uniform distribution on {\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}} with \hat{\pi}_{s}={\color[rgb]{0,0,0}\hat{S}}_{N,\mathbf{d}}^{-1} and {\color[rgb]{0,0,0}\hat{S}}_{N,\mathbf{d}}=|{\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}|. The configuration ensemble of simple random networks is defined by the pair {\color[rgb]{0,0,0}\hat{\mathbf{G}}}_{N,\mathbf{d}}=({\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}},\hat{\pi}). For further clarity, we use hats on not-already-accented mathematical objects associated with the configuration ensemble to clearly delineate those objects from analogous objects for the ER ensemble. *

There are two main classes of algorithms for sampling random networks from {\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}: random-matching methods and Markov chain Monte Carlo (MCMC) methods [11, 50]. The random-matching methods involve enumerating the $d_{i}$ “stubs” of edges for each node $i$ and then randomly matching pairs of stubs of different nodes in a “configuration” of allowable network edges [8, 13, 76]. We note that only some degree sequences $\mathbf{d}$ are graphical in that there exist graphs for such a degree sequence [4]. In contrast, the MCMC methods involve taking an initial network and randomizing it via repeated stochastic rewiring, which can be studied as a Markov chain in which states represent networks and transitions represent rewiring [22, 38] (see Fig. 2). In general, there are many choices for how to implement stochastic rewiring, which can give rise to various random-network ensembles. We describe in the next section an MCMC rewiring process that converges in the limit of many rewires to uniform sampling [28] (consistent with Definition 2.5).

Finally, we note that there exist many other generative models for constructing random networks—stochastic block models [75, 64], exponential random graphs [42], and so on (see reviews [12, 37, 57] and references therein)—that introduce different constraints aimed toward diverse applications.

2.4 Degree-Preserved Rewiring as a Markov Chain on a Set of Networks

We now describe a stochastic rewiring process called double-edge-swap vertex-labeled rewiring [28] that can be used for MCMC sampling of the configuration ensemble given by Definition 2.5. Herein, we refer to the process as degree-preserved rewiring.

We note that this rewiring process has been unknowingly reinvented several times by different researchers from different fields, and this work is reviewed and expanded on in [28].

Definition 2.6** (Degree-Preserved Rewiring [28]).**

*Degree-preserved rewiring is a stochastic map T_{DP}:{\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}\to{\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}} defined by $(\mathcal{V},\mathcal{E})\to(\mathcal{V},\mathcal{E}^{\prime})$, where $\mathcal{E}^{\prime}$ is given by the following stochastic process. Choose two unique edges $(i,j)$ and $(i^{\prime},j^{\prime})$ uniformly at random from $\mathcal{E}$. Consider a proposed edge swap in which two edges $\mathcal{E}^{(-)}=\{(i,j),(i^{\prime},j^{\prime})\}$ are removed and two new edges $\mathcal{E}^{(+)}$ are added, uniformly at random selecting between $\mathcal{E}^{(+)}=\{(i,j^{\prime}),(i^{\prime},j)\}$ and $\mathcal{E}^{(+)}=\{(i,i^{\prime}),(j^{\prime},j)\}$. If the proposed new edges $\mathcal{E}^{(+)}$ do not give rise to a self edge nor a repeat edge (which are disallowed by {\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}), then the proposed edge swap is implemented, $\mathcal{E}\mapsto(\mathcal{E}\setminus\mathcal{E}^{(-)})\cup\mathcal{E}^{(+)}$. Otherwise, the network is left unchanged, $\mathcal{E}\mapsto\mathcal{E}$. *

Because $N$ and $\mathbf{d}$ are invariant under degree-preserved rewiring, iterative rewiring can be modeled as a random walk on the set {\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}=\{\hat{G}_{s}\}, which we enumerate using s\in\{1,\dots,{\color[rgb]{0,0,0}\hat{S}}_{N,\mathbf{d}}\}. We let \hat{{G}}^{(0)}\in{\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}} denote an original (e.g., empirical) network, {\color[rgb]{0,0,0}\hat{G}}^{(t)}\in{\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}} denote a network after $t$ rewires, $\hat{\pi}_{s}^{(t)}$ denote the probability that $\hat{{G}}^{(t)}=\hat{{G}}_{s}$. Note that ${\bf\hat{\pi}}^{(0)}_{s}=1$ for $s$ such that {\color[rgb]{0,0,0}\hat{G}}^{(0)}={\color[rgb]{0,0,0}\hat{G}}_{s} and ${\bf\hat{\pi}}^{(0)}_{s}=0$ otherwise. The evolution of ${\bf\hat{\pi}}^{(t)}$ is given by

[TABLE]

where $\hat{P}_{rs}$ is a transition matrix describing the probability that network {\color[rgb]{0,0,0}\hat{G}}_{r} will become {\color[rgb]{0,0,0}\hat{G}}_{s} after a degree-preserved rewire.

Theorem 2.7** (Markov Chain Convergence for Degree-Preserved Rewiring [28]).**

The Markov chain defined by Eq. 5 is ergodic and has uniform stationary distribution

[TABLE]

Proof 2.8**.**

*The result follows from showing that the Markov chain is connected, aperiodic and degree regular [28]. *

Remark 2.9**.**

*See [34, 35] for research relating the rate of convergence to properties of the set of degrees that is preserved under rewiring. *

3 Main Results

Our approach for efficient network-ensemble comparison involves using a perturbation analysis of stochastic rewiring processes associated with random-network ensembles. In Sec. 3.1, we define and analyze a uniform stochastic-rewiring process. In Sec. 3.2, we analyze the distributional convergence of network summary statistics (including VNE) for networks undergoing stochastic rewiring. In Sec. 3.3, we develop a first-order perturbation analysis for the effect of rewiring on VNE. In Sec. 3.4, we study the expected perturbation to VNE and the Laplacian matrix under uniform rewiring. In Sec. 3.5, we provide a roadmap for how this approach can be extended to other rewiring processes and other summary statistics.

3.1 Convergence of Network Ensembles obtained under Stochastic Rewiring

In this section, we define and study convergence for a sequences $\{\mathbf{G}^{(t)}\}$ of random-network ensembles arising from two stochastic rewiring processes. We begin by defining another stochastic rewiring process: uniform rewiring.

Definition 3.1** (Uniform Rewiring).**

*Uniform rewiring is a stochastic map $T_{U}:\mathcal{G}_{N,M}\to\mathcal{G}_{N,M}$ defined by $(\mathcal{V},\mathcal{E})\to(\mathcal{V},\mathcal{E}^{\prime})$, where $\mathcal{E}^{\prime}$ is given by the following stochastic process. Choose uniformly at random an edge $(i,j)\in\mathcal{E}$ and remove it from $\mathcal{E}$. Then choose uniformly at random a new edge $(i^{\prime},j^{\prime})$ from the $N(N-1)/2-M+1$ possible edges outside of $\mathcal{E}\setminus(i,j)$ [that is, allowing re-selection of $(i,j)$], and add the edge to $\mathcal{E}$. It follows that $\mathcal{E}\mapsto(\mathcal{E}\setminus(i,j))\cup(i^{\prime},j^{\prime})$. *

Because $N$ and $M$ are invariant under uniform rewiring, iterative uniform rewiring can be modeled as a random walk on the set $\mathcal{G}_{N,M}=\{{G}_{s}\}$, which we enumerate using $s\in\{1,\dots,S_{N,M}\}$. We let $G^{(t)}\in\mathcal{G}_{N,M}$ denote a network after it undergoes $t$ rewires and ${\pi}_{s}^{(t)}$ denote the probability that {\color[rgb]{0,0,0}G}^{(t)}={\color[rgb]{0,0,0}G}_{s}. Obviously, ${\bf{\pi}}^{(0)}_{s}=1$ for $s$ such that {{\color[rgb]{0,0,0}G}}^{(0)}={{\color[rgb]{0,0,0}G}}_{s} and ${\bf{\pi}}^{(0)}_{s}=0$ otherwise. The evolution of ${\bf{\pi}}^{(t)}$ is given by

[TABLE]

where ${P}_{rs}$ is a transition matrix describing the probability that network $G_{r}$ will become $G_{s}$ after a uniform rewire.

We identify the following limiting behavior for ${\bf\pi}^{(t)}$.

Theorem 3.2** (Convergence of Uniform Rewiring).**

The Markov chain for uniform rewiring (7) is ergodic and has uniform stationary distribution

[TABLE]

Proof 3.3**.**

*See Appendix A. *

We now define a notion of convergence for a sequence $\{\mathbf{G}^{(t)}\}$ of random-network ensembles.

Definition 3.4** (Convergence of Random-Network Ensembles).**

Let $\{\mathbf{G}^{(t)}\}$ denote a sequence of random-network ensembles in which $\mathbf{G}^{(t)}=(\{G_{s}\},\pi^{(t)})$. We say that $\mathbf{G}^{(t)}$ converges to $\mathbf{G}=(\{G_{s}\},\pi)$ iff

[TABLE]

We are now ready to describe the convergence of random-network ensembles arising from stochastic uniform and degree-preserved rewiring.

Corollary 3.5** (ER Ensemble Convergence).**

Consider the ensemble of random networks $\mathbf{G}^{(t)}=(\mathcal{G}_{N,M},\pi^{(t)})$ obtained by $t$ uniform rewires of an initial network $G^{(0)}\in\mathcal{G}_{N,M}$. The sequence $\{\mathbf{G}^{(t)}\}$ of ensembles converges to the ER ensemble given by Definition 2.4,

[TABLE]

Proof 3.6**.**

*The result follows straightforwardly from Theorem 3.2. *

Corollary 3.7** (Configuration-Model Ensemble Convergence).**

Consider the ensemble of random networks \hat{\mathbf{G}}^{(t)}=({\color[rgb]{0,0,0}\hat{{\mathcal{G}}}}_{N,\mathbf{d}},\hat{\pi}^{(t)}) that is obtained by $t$ degree-preserved rewires of an initial network \hat{G}^{(0)}\in{\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}. The sequence $\{\hat{\mathbf{G}}^{(t)}\}$, where \hat{\mathbf{G}}^{(t)}=(\{\hat{G}_{s}\},{\color[rgb]{0,0,0}\hat{\pi}}^{(t)}), converges to the configuration model ensemble given by Definition 2.5,

[TABLE]

Proof 3.8**.**

*The result follows straightforwardly from Theorem 2.7. *

3.2 Distributional Convergence of Network Summary Statistics

We now study the distribution of VNE and other summary statistics for random-network ensembles associated with uniform and degree-preserved rewiring.

Theorem 3.9** (Distributional Convergence of Network Statistics).**

Let $\{\pi^{(t)}\}$ and $\{{\hat{\pi}}^{(t)}\}$ describe sequences of probability distributions over $\mathcal{G}_{N,M}$ and {\color[rgb]{0,0,0}\hat{\mathcal{G}}}_{N,\mathbf{d}}, respectively, for the uniform and degree-preserved rewiring processes. Further, let $f:\{G_{s}\}\mapsto\mathbb{R}$ denote any scalar-valued function on a network and let

[TABLE]

denote the respective distributions of $f$ across the associated random-network ensembles $\mathbf{G}^{(t)}$ and $\hat{\mathbf{G}}^{(t)}$. Here, $\delta_{g}(f)$ is the Dirac delta function with weight concentrated at $f=g$ [i.e., $\delta_{g}(f)=\delta(f-g)$]. The following limits converge in distribution as $t\to\infty$

[TABLE]

where $P^{(N,M)}(f)$ and {\color[rgb]{0,0,0}\hat{P}}^{(N,\mathbf{d})}(f) denote the distributions of $f(G)$ for the ER and configuration random-network ensembles, respectively.

Proof 3.10**.**

*We take the limit of both sides of the equations in (12). Because the summations are finite, the limits can be taken inside the summation. The equations in (3.9) follow directly from Eqs. (3.5) and (3.7). *

Corollary 3.11** (Distributional Convergence of VNE).**

Letting $f:\{G_{s}\}\mapsto\mathbb{R}$ denote VNE, $h(G_{s})$, given by Definition 2.1, Eq. (3.9) implies

[TABLE]

where $P^{(N,M)}(h)$ and {\color[rgb]{0,0,0}\hat{{P}}}^{(N,\mathbf{d})}(h) denote the distributions of VNE for the ER and configuration random-network ensembles, respectively.

3.3 Perturbation of VNE and Laplacian Matrices

Having characterized the long-time behavior (i.e., after many rewires) of networks subjected to uniform and degree-preserving rewiring processes (as well as their associated network statistics such as VNE), we now turn our attention to studying the effect on VNE due to a small number of rewires. To this end, in this section we develop a first-order perturbation analysis for VNE. We begin by presenting a well-known result that describes the first-order perturbation of eigenvalues and eigenvectors of a symmetric matrix, which we present for an unnormalized Laplacian ${L}$.

Theorem 3.12** (Perturbation of Simple Eigenvalues and their Eigenvectors [5]).**

Let ${L}$ be a symmetric $N\times N$ matrix with simple eigenvalues $\{\lambda_{i}\}$ and normalized eigenvectors $\{\bm{v}^{(i)}\}$. Consider a fixed symmetric perturbation matrix $\Delta{L}$, and let ${L}(\epsilon)={L}+\epsilon\Delta{L}$. We denote the eigenvalues and eigenvectors of ${L}(\epsilon)$ by $\lambda_{n}(\epsilon)$ and $\bm{v}^{(n)}(\epsilon)$, respectively, for $i=1,2,\dots,N$. It follows that

[TABLE]

and the derivatives with respect to $\epsilon$ at $\epsilon=0$ are given by

[TABLE]

Proof 3.13**.**

*See [5]. *

Remark 3.14**.**

*For the unnormalized Laplacian matrix $L$, $\lambda_{1}(\epsilon)=0$ and $\bm{v}^{(1)}(\epsilon)=N^{1/2}\bm{1}$ for all values of $\epsilon$. Any allowable perturbation matrix $\Delta{L}$ will have the same null space as ${L}$, $\text{span}(\bm{1})$, and so $\lambda_{1}^{\prime}(0)=0$ and ${\bm{v}^{\prime}}^{(1)}(0)=\bm{0}$. *

Remark 3.15**.**

*The first-order approximations in Eq. (15) are accurate when the perturbations are small. However, the regime for which this approximation is valid (i.e., how small $\epsilon$ needs to be) generally depends on ${L}$, $\epsilon$, and the perturbation $\Delta{L}$. Accuracy typically requires $\epsilon\lambda^{\prime}_{i}(0)/\lambda_{i}$ to be small [80]. *

We now present a first-order perturbation analysis of the VNE for a network subjected to a modification.

Theorem 3.16** (First-Order Perturbation of VNE).**

Let $h(0)$ denote the VNE given by Definition 2.1 for an unnormalized network Laplacian ${L}$ with simple eigenvalues $\{\lambda_{i}\}$, and let $h(\epsilon)$ denote the VNE for the network after it undergoes a network modification encoded by ${L}(\epsilon)={L}+\epsilon\Delta{L}$. We assume the eigenvalues of ${L}(\epsilon)$ are simple. The first-order expansion in $\epsilon$ for the perturbed VNE is

[TABLE]

where

[TABLE]

Proof 3.17**.**

*See Appendix B *

Remark 3.18**.**

*The Laplacian matrix $L$ for networks consisting of $k$ connected components will have $K$ eigenvalues $\lambda_{1}=\lambda_{2}=\dots=\lambda_{k}=0$. In this case, as well as other scenarios with eigenvalues having multiplicity greater than or equal to two, Eqs. (15)–(18) can be used to estimate the perturbation of the remaining simple eigenvalues that have multiplicity one, and for which $\frac{1}{\lambda_{i}-\lambda_{j}}$ is guaranteed to be finite. *

Corollary 3.19** (Edge Perturbation of von Neumann Entropy).**

When the network is modified by adding (+) or removing (-) an unweighted edge $(p,q)$, the Laplacian perturbation matrix takes the form

[TABLE]

and Eq. (18) can be simplified as

[TABLE]

where $\pm$ corresponds to addition and removal, respectively.

Proof 3.20**.**

*See Appendix C *

Corollary 3.21** (Edge-Set Perturbation of von Neumann Entropy).**

When the network is modified by adding a set of edges $\mathcal{E}^{+}$ and removing a set $\mathcal{E}^{-}$, the Laplacian perturbation matrix takes the form

[TABLE]

where $\Delta{L}_{ij}^{(pq)}$ is given by Eq. (22), and Eq. (18) becomes

[TABLE]

Proof 3.22**.**

*The proof is straightforward using the linearity property of edge additions and removals. *

3.4 Expected Perturbations under Uniform Rewiring

In this section, we describe the expected perturbations due to uniform rewiring. We note that solving the expected perturbations under degree-preserved stochastic rewiring is much more difficult and is left for future research.

Theorem 3.23** (Expected Change to the Laplacian under Uniform Rewiring).**

Consider an undirected unweighted network {\color[rgb]{0,0,0}G} with $N$ nodes, $M$ edges, adjacency matrix $A$, node degrees $\{d_{i}\}$, and Laplacian matrix $L$. The expected change $\Delta L$ to $L$ under uniform rewiring (see Definition 3.1) is given by

[TABLE]

Proof 3.24**.**

*See Appendix D *

Corollary 3.25** (Expected First-Order Perturbation under Uniform Rewiring).**

Under uniform rewiring (see Definition 3.1), the expected first-order terms for $\lambda_{i}(\epsilon)$ and ${\bm{v}^{(i)}}(\epsilon)$ [see Eq. (16)] and $h(\epsilon)$ [see Eq. (18)] are given by

[TABLE]

where $\mathbf{E}[\Delta L]$ is given by Eq. (26).

Proof 3.26**.**

*We take the expectation of Eqs. (16) and (18), use the linearity property for expectation, and combine these results with Eq. (26). *

3.5 Road Map for Other Rewiring Processes and Other Summary Statistics

The analytical approximations given by Thm. 3.23 and Corr. 3.25 could be obtained due to the simplicity of uniform rewiring (see Definition 3.1). However, this is not the case for degree-preserved rewiring (see Definition 2.6), for which analytical predictions for $\mathbf{E}[\Delta L_{ij}]$ and ${\bf E}[h^{\prime}(0]$ are beyond the scope of this paper. A main difficulty stems from the observation that the probability of rewiring a given edge $(a,b)$ depends on whether or not it is swapped with another edge $(c,d)$. Given a pair of edges, $\{(a,b),(c,d)\}$, the probability of swapping $\{(a,b),(c,d)\}\to\{(a,d),(b,c)\}$ or $\{(a,b),(c,d)\}\to\{(a,c),(b,d)\}$, or not swapping at all, depends on whether or not the edges $(a,c)$, $(a,d)$, $(b,c)$, and $(b,d)$ already exist. Moreover, for a given edge $(a,b)$, the expected change $\mathbf{E}[\Delta L_{ab}]$ requires considering the probability of swapping with all other edges $\{(c,d)\}$, and this pursuit quickly becomes intractable for large networks. Developing perturbation theory for degree-preserved rewiring (and other stochastic rewiring processes [28]) remains an important direction for future research.

Nevertheless, by iteratively simulating one single rewire, one can still numerically estimate ${\bf E}[h^{\prime}(0)]$ (or the expected change to other summary statistics) for degree-preserved rewiring and other rewiring processes. In the following section, we will make use of these analytical and numerical predictions for ${\bf E}[h^{\prime}(0)]$ to estimate the number of rewires necessary for empirical networks to resemble typical networks from ER and configuration-model network ensembles.

4 Numerical Experiments

We now present numerical experiments supporting and demonstrating the utility of our results from Sec. 3. In Sec. 4.1, we support our perturbation results describing how network modifications affect VNE. In Sec. 4.2, we support our results for the distributional convergence of VNE for stochastic uniform and degree-preserved rewiring processes. In Sec. 4.3, we highlight an application of our analysis: network-ensemble comparison for empirical networks.

4.1 Perturbation Results

We first provide numerical validation for the first-order approximation given by Eq. (17), which estimates how a network modification encoded by the perturbed Laplacian matrix $\Delta L$ affects VNE. We created a random ER network with $N=1000$ nodes and $M=50,000$ edges, and subjected it to iterative uniform rewiring. We denote the original network {\color[rgb]{0,0,0}G^{(0)}} and the network after $t$ steps of uniform rewiring by {\color[rgb]{0,0,0}G^{(t)}}, and we use $L^{(t)}$ and $h_{t}=h(G^{(t)})$ to denote the respective Laplacian matrices and VNEs for each $t=0,1,2,\dots$. In Fig. 3(a), we compare the true values of $\{h_{t}\}$ of the rewired network with predicted values using the first-order approximation given by Eq. (17) for $K=1$ trial of uniform rewiring. These are in very good agreement for small $t$. We point out that the first-order approximation is expected to improve in accuracy as the eigenvalues $\{\lambda_{i}\}$ become larger, which typically occurs as $N$ and $M$ increase. We note that the first-order approximations described in Sec. 3.3 can become inaccurate when $N$ and $M$ are too small.

In the next experiment, we support the results of Sec. 3.4 in which we analyze the expected changes $\mathbf{E}[\Delta L]$ and $\mathbf{E}[\Delta h]$ under uniform rewiring. We created an ER network with $N=100$ nodes and $M=1,000$ edges, and subjected it to $K=10,000$ trials of iterative uniform rewiring. In Fig. 3(b), we compare the empirical mean ${\mathbf{E}}[\Delta L_{ij}^{(t)}]$ (symbols) to its expectation given by Theorem 3.23 (dashed lines). We make the comparison for three entries: $\Delta L_{ii}$ for a diagonal entry, $\Delta L_{ij}$ for an entry in which $(i,j)$ is an edge in {\color[rgb]{0,0,0}G^{(0)}}, and $\Delta L_{ij}$ for an entry in which $(i,j)$ is not an edge in {\color[rgb]{0,0,0}G^{(0)}}. Error bars indicate standard error.

4.2 Distributional Convergence of VNE

In Sec. 3.2, we showed that the distribution of VNEs across the ensemble of networks obtained after iterative stochastic uniform and degree-preserved rewires converges, respectively, to the distribution of VNEs across the ER and configuration random-network ensembles. Here, we support this result by studying uniform rewiring for an empirical network: an adjacency network of words in the novel David Copperfield by Charles Dickens [56]. The network contains $N=112$ nodes (which represent the adjectives and nouns with highest frequency in the book) and $M=425$ edges (which represent a pair of words that occur adjacent to one another).

We study how stochastic rewiring affects the VNE of this network by considering the distribution $P^{(t)}(h)$ of VNEs across networks {\color[rgb]{0,0,0}G^{(t)}} obtained after $t$ rewires. In Fig. 4(a), we show by solid curves the empirical distributions $P^{(t)}(h)$ for several values of $t\in\{0,10,100,1000\}$. The distributions are estimated using $K=1,000$ trials of rewiring for each $t$. Note that at time $t=0$, $P^{(0)}(h)=\delta_{h_{0}}(h)$ is a Dirac delta function at $h_{0}=h(G^{(0)})\doteq 6.277$. As $t$ increases, $P^{(t)}(h)$ widens and shifts to the right and eventually converges to $P^{(N,M)}(h)$, the distribution of VNE for the corresponding ER ensemble $\mathbf{G}_{N,M}$ (estimated using $K=10,000$ sample ER networks and shown by the dashed curve).

In Fig. 4(b), we further study the convergence of $P^{(t)}(h)\to P^{(N,M)}(h)$ by plotting the 5% quantile, mean and 95% quantile of ${P}^{(t)}(h)$. These respectively converge to the 5% quantile, mean and 95% quantile for the distribution $P^{(N,M)}(h)$. The horizontal solid line indicates the mean VNE across $\mathbf{G}_{N,M}$ given by

[TABLE]

We define the $\alpha$-quantile of $P^{(N,M)}(h)$ by

[TABLE]

and we plot {\color[rgb]{0,0,0}H^{(N,M)}}(0.05) and {\color[rgb]{0,0,0}H^{(N,M)}}(0.95) by horizontal dashed lines.

These quantiles were numerically approximated for $K=10,000$ sample ER networks. Numerically estimating $H^{(N,M)}(\alpha)$ is obviously associated with a computational cost, which depends on the accuracy required by the application. We note, however, that any network-ensemble comparison using network summary statistics requires knowledge about how the summary statistic varies across the network ensemble. This highlights the need for further theory development for the distribution of VNE (and other summary statistics) across network ensembles.

Returning to Fig. 4(b), because $h(G^{(0)})\doteq 6.277$ is much smaller than the typical VNE values for the ensemble, the empirical network is much more irregular than is typical for the ensemble. Moreover, one can observe in Fig. 4(b) that the distribution $P^{(t)}(h)$ obtained after $t=1000$ uniform rewires closely resembles the distribution $P^{(N,M)}(h)$. We can therefore conclude from Fig. 4 that the word-adjacency network is atypical for the ER random-network ensemble. In the next section, we more rigorously describe how to use the VNE of stochastically rewired networks to study and quantify network-ensemble comparisons.

4.3 Network-Ensemble Comparisons for Empirical Networks

Given the convergence $P^{(t)}(h)\to P^{(N,M)}(h)$, there are many different ways to define and quantify what it means for a network to “resemble a typical network.” For example, one could ask how many rewires are necessary for $\overline{h}^{(t)}$, the mean VNE of a network obtained after $t$ rewires, to be within some range of the ensemble mean, $\overline{h}^{(N,M)}$. Or one could measure the smallest $t$ such that \overline{h}^{(t)}\in[{\color[rgb]{0,0,0}H^{(N,M)}}(\alpha),{\color[rgb]{0,0,0}H^{(N,M)}}(1-\alpha)]\subset\mathbb{R}, where {\color[rgb]{0,0,0}H^{(N,M)}}(\alpha) is the $\alpha$-quantile given by Eq. (31). Another possibility is to ask how many rewires are necessary (on average) for $h_{t}=h(G^{(t)})$ of a rewired network $G^{(t)}$ to first fall within this range—that is, the mean hitting time

[TABLE]

Given that a stochastic rewiring process can be modeled as a random walk on a set of networks, $\tau_{\alpha}$ is equivalent to the mean first-passage time for a random walk that starts at network {\color[rgb]{0,0,0}G^{(0)}} and reaches the subset of networks \{{G}_{s}\in\mathcal{G}_{N,M}:h(G_{s})\in[{\color[rgb]{0,0,0}H^{(N,M)}}(\alpha),{\color[rgb]{0,0,0}H^{(N,M)}}(1-\alpha)]\}. Unfortunately, these methods are computationally expensive in that they require one to simulate $t\gg 1$ stochastic rewires across $K\gg 1$ independent trials of rewiring, all while computing the VNE for the many rewired network realizations.

Thus motivated, we propose a computationally efficient technique for network-ensemble comparison that does not require computing the VNE of rewired networks. In fact, it avoids simulating stochastic rewires altogether. Instead, we introduce a quantity that utilizes our first-order perturbation analysis of Sec. 3.4.

Definition 4.1** (Exponential Extrapolation for $\alpha$-Quantile Intersect).**

Suppose we fit an exponential model to $h_{t}$, which has the following form

[TABLE]

The model satisfies the conditions that when $t=0$ we have $\tilde{h}(0)=h_{0}$, $\tilde{h}^{\prime}(0)=\mathbf{E}[h^{\prime}(0)]$, and $\tilde{h}(t)\to\overline{h}^{(N,M)}$ as $t\to\infty$.

By solving the equation

[TABLE]

we obtain

[TABLE]

*where {\color[rgb]{0,0,0}H^{(N,M)}}{(\alpha)} is given by Eq. (31), $\overline{h}^{(N,M)}$ is given by Eq. (30) and $\mathbf{E}[h^{\prime}(0)]$ is given by Eq. (29). Note that for $B_{\alpha}$ to be properly defined, (\overline{h}^{(N,M)}-{\color[rgb]{0,0,0}H^{(N,M)}}(\alpha)) must have the same sign as $(\overline{h}^{(N,M)}-h_{0})$. For example, if $h_{0}<\overline{h}^{(N,M)}$, then $\alpha$ could be $0.05$ but not $0.95$. *

The quantity $B_{\alpha}$ is an exponential extrapolation that estimates the number $t$ of uniform rewires required to modify a given network so that {\color[rgb]{0,0,0}h_{t}} falls between the $\alpha$ and $(1-\alpha)$ quantiles of $P^{(N,M)}(h)$. For example, returning to the experiment described in Sec. 4.2, the blue arrow in Fig. 4(b) indicates the initial slope $\mathbf{E}[h^{\prime}(0)]$, and the blue star indicates the intersection point (B_{\alpha},{\color[rgb]{0,0,0}H^{(N,M)}}(\alpha)) for $\alpha=0.05$ at which \tilde{h}(B_{\alpha})={\color[rgb]{0,0,0}H^{(N,M)}}(\alpha).

We now study network-ensemble comparisons for the empirical networks described in Table 1. In Fig. 5(a), we compare $B_{\alpha}$ given by Eq. (35) to the mean first-passage time $\tau_{\alpha}$ given by Eq. (32) for the associated ER ensemble. For each empirical network, we perform $K=10$ independent simulations of repeated uniform rewiring, which we iterate until $h_{t}\in[H(\alpha),H(1-\alpha)]$ (the “hitting” criterion). The $\alpha$-quantiles of $P^{(N,M)}(h)$ are estimated using $K=100$ samples from the ER random-network ensemble. We observe a strong linear correlation between $B_{\alpha}$ and $\tau_{\alpha}$. In fact, the results fall along a diagonal line indicating $\tau_{\alpha}=B_{\alpha}$. In Fig. 5(b), we compare $B_{\alpha}$ to two other quantities: (i) the difference $\Delta h=h(G^{(0)})-\overline{h}^{(N,M)}$ in VNE between the original network and the ensemble mean $\overline{h}^{(N,M)}$ as well as (ii) the ratio of $\Delta h$ to the standard deviation $\sigma_{N,M}$ of $P^{(N,M)}(h)$. Interestingly, these two other quantities do not as strongly correlate with $B_{\alpha}$ and $\tau_{\alpha}$. Finally, we note that the result for the power grid in Fig. 5 does not use the first-order approximation for ${\bf E}[h^{\prime}(0)]$ given by Eq. (29). We find that it is not accurate for this network, and in fact has the wrong sign, which causes the exponential extrapolation given by Eq. (33) to diverge rather than converge. (Interestingly, the power grid is the only network in Table 1 for which $h_{0}>\overline{h}^{(N,M)}$ and ${\bf E}[h^{\prime}(0)]<0$.) Therefore, for this network we used a numerical estimate for ${\bf E}[h^{\prime}(0)]$ based on the of VNE change $(h_{1}-h_{0})$ due to a single uniform rewire, which we averaged across 1000 simulations (see discussion in Sec. 3.5).

In Fig. 6, we present similar results as those shown in Fig. 5, except we now compare the empirical networks from Table 1 to configuration-model ensembles containing networks with the same degree sequence. We estimated $\hat{H}^{(N,{\bf d})}(\alpha)$—which is given by Eq. (31) under the variable substitution $\hat{P}^{(N,{\bf d})}(h)\mapsto P^{(N,M)}(h)$—by sampling the configuration-model ensemble using degree-preserved rewiring. Starting with the associated empirical network, we implemented $10^{5}$ degree-preserved rewires. Then, we implemented $10^{5}$ additional rewires, sampling the network’s VNE every $100$ rewires, which allowed us to estimate $\hat{P}^{(N,{\bf d})}(h)$ and $\hat{H}^{(N,{\bf d})}(\alpha)$ using $K=1000$ VNE samples. To numerically estimate ${\bf E}[h^{\prime}(0)]$ (see Sec. 3.5), we simulated $1000$ degree-preserved rewires, each time modifying the original network by a single rewire, and computed mean difference $(h_{1}-h_{0})$ across the simulations. Using this estimate for ${\bf E}[h^{\prime}(0)]$, we obtained an exponential extrapolation for the intersect of $h_{t}$ with $\hat{H}^{(N,{\bf d})}(\alpha)$ using Eq. (35) with the variable substitutions $\overline{h}^{(N,{\bf d})}\mapsto\overline{h}^{(N,{M})}$ and $\hat{H}^{(N,{\bf d})}\mapsto H^{(N,{M})}$.

In Fig. 6(A), we compare $\hat{B}_{\alpha}$ to an empirical mean hitting time $\hat{\tau}_{\alpha}$ given by Eq. (32) under the variable substitution $\hat{H}^{(N,{\bf d})}\mapsto H^{(N,{M})}$. In Fig. 6(B), we show that $\Delta h=h(\hat{G}^{(0)})-\overline{h}^{(N,{\bf d})}$ and $\Delta h/\hat{\sigma}_{N,{\bf d}}$, where $\hat{\sigma}_{N,{\bf d}}$ is the empirical standard deviation of $\hat{P}^{(N,{\bf d})}(h)$, do not strongly correlate with $\hat{B}_{\alpha}$ (or $\hat{\tau}_{\alpha}$). It can observed that the empirical networks generally require fewer stochastic rewires to obtain typical VNE values for the configuration-model ensembles than was required for the ER ensembles (recall Fig. 5). Interestingly, the word-adjacency network [56] (see also Fig. 4) is omitted from Fig. 6(A), because it was found to lie within the $\alpha=0.05$ and $\alpha=0.95$ quantiles of $\hat{P}^{(N,{\bf d})}(h)$, indicating that it closely resembles a typical configuration-model graph according to the VNE statistic.

5 Discussion

We have studied the von Neumann Entropy (VNE) of networks subjected to two stochastic rewiring processes: uniform and degree-preserved rewiring. We presented our main mathematical results in Sec. 3. First, we proved that the network-ensemble given by networks obtained through iterative uniform rewiring converges to the Erdős-Rényi ensemble $\mathbf{G}_{N,M}$ of simple networks. Next, we proved that the distribution of network summary statistics for networks obtained from iterative uniform and degree-preserved rewiring converge to their respective distributions associated with the Erdős-Rényi $\mathbf{G}_{N,M}$ and configuration {\color[rgb]{0,0,0}\hat{\mathbf{G}}}_{N,\bf{d}} ensembles (offering insight toward network-ensemble comparisons). We also conducted a perturbation analysis for how rewiring affects VNE, offering insight toward network-network comparisons. In particular, we obtained a first-order approximation for the expected change in VNE after $t$ uniform rewires.

In Sec. 4, we showed that the study of VNE for an empirical network subjected to repeated uniform rewires allows one to assess how many rewires are required before the rewired networks obtain VNE values that are typical for the $\mathbf{G}_{N,M}$ and $\hat{\mathbf{G}}_{N,{\bf d}}$ ensembles. Importantly, such a numerical study can be computationally infeasible since it can require simulating many steps of rewiring, many independent trials of rewiring, and repeated calculations of VNE for the rewired networks. Thus motivated, we introduced a computationally efficient quantity $B_{\alpha}$ to quantify network-ensemble comparisons. It combines our perturbation and convergence analyses and is based on an exponential extrapolation for when the VNE of rewired networks intersects an $\alpha$ quantile of the VNE distribution $P(h)$ for the appropriate ensemble (see Fig. 4). The quantity $B_{\alpha}$ is computationally efficient since it does not require iteratively rewiring a network nor recomputing VNE for these networks. In the case of $\mathbf{G}_{N,M}$, $B_{\alpha}$ used an analytical approximation for the perturbative influence of uniform rewiring on VNE (see Corr. 3.25). In the case of $\hat{\mathbf{G}}_{N,{\bf d}}$, $\hat{B}_{\alpha}$ required a numerical estimate for the perturbative influence of degree-preserved rewiring on VNE, which provides a road map for how to extend this methodology for other random-network ensembles [12, 37, 57] and other stochastic rewiring processes [28]. Because the approach of linearizing the effect of stochastic rewires to estimate network-ensemble comparisons is computationally efficient, future work should explore its application to model selection and hypothesis testing [7, 16, 42, 51].

To our knowledge, this is the first use of VNE for network-ensemble comparison. We have focused on VNE due to growing interest in VNE-based network-network comparisons, such as clustering network layers in multilayer networks [24, 23]. We point out, however, that our mathematical techniques—specifically, the approach of linearizing the effect of rewiring so as to obtain an exponential extrapolation $B_{\alpha}$—can be extended to study convergence and assess network-ensemble comparisons through other network summary statistics (e.g., degree distribution, size, clustering coefficient, and so on). For example, it would be interesting to extend our work to a complementary definition for VNE that was recently introduced [23]. Because one can numerically approximate the linear effect of stochastic rewiring on a summary statistic (e.g., see the results in Fig. 6 and discussion in Sec. 3.5), our approach should be widely generalizable to other network summary statistics as well as vectors of statistics. This approach may shed light toward the relation between different summary statistics and identify which ones are more meaningful to different network ensembles, whether it’s basic statistics, information-theoretic measures such as VNE (e.g., Eq. (3) or otherwise [23]), or some new summary statistic yet to be discovered.

Appendix A Proof of Theorem 3.2

Proof A.1**.**

The result follows from showing the Markov chain is connected, aperiodic and degree regular.

We first prove the Markov chain described in Eq. (7) corresponds to a connected graph. To this end, we will show for any two networks there exists a path—that is, a sequence of edge swaps allowed by uniform rewiring—between the two networks. Let $G_{s}=(\mathcal{V},\mathcal{E}_{s})\in\mathbf{G}_{N,M}$ and $G_{r}=(\mathcal{V},\mathcal{E}_{r})\in\mathbf{G}_{N,M}$ and define $\Delta^{(s)}=\mathcal{E}_{s}\setminus\mathcal{E}_{r}$ and $\Delta^{(r)}=\mathcal{E}_{r}\setminus\mathcal{E}_{s}$ indicate, respectively, the set of edges in $\mathcal{E}_{s}$ and $\mathcal{E}_{r}$ that are not in the other edge set. Because $M=|\mathcal{E}_{s}|=|\mathcal{E}_{r}|$, it follows that $T=|\Delta^{(s)}|=|\Delta^{(r)}|$. We enumerate the entries in $\Delta^{(s)}$ and $\Delta^{(r)}$ as $\Delta^{(s)}_{j}$ and $\Delta^{(r)}_{j}$ for $j=\{1,\dots,T\}$ and define the family of maps $T_{r,s}^{(j)}:\mathbf{G}_{N,M}\mapsto\mathbf{G}_{N,M}$ by (\mathcal{V},\mathcal{E})\mapsto\big{(}\mathcal{V},(\mathcal{E}\setminus\Delta_{j}^{(r)})\cup\Delta_{j}^{(s)}\big{)}. It follows that

[TABLE]

We can similarly define $T_{s,t}^{(j)}$ so that

[TABLE]

Next, we prove the graph is degree-regular. Consider a network $G_{s}\in\mathcal{G}_{N,M}$ containing $N$ nodes and $M$ edges. It follows that there are $M$ possibilities for edge removal and $N(N-1)/2-M+1$ possibilities for new edges to add. (Here, the $+1$ allows for the removed edge to be replaced.) Moreover, for any transition $G_{s}\to G_{r}$, there exists a transition $G_{r}\to G_{s}$ with identical rate. Therefore, the Markov chain corresponds to an undirected network in which all nodes have degree $d_{U}=M(N(N-1)/2-M+1)$.

Finally, we prove aperiodicity. Because the graph is connected and contains self-edges, the Markov chain is aperiodic.

Appendix B Proof of Theorem 3.16

Proof B.1**.**

Taylor expansion near $\epsilon=0$ gives

[TABLE]

Here we show that $h^{\prime}(0)$ is given by Eq. (18). Using Eq. (4), the VNE entropy of a network corresponding to Laplacian matrix $L(\epsilon)$ is given by

[TABLE]

where

[TABLE]

has derivative

[TABLE]

Using the linearity property of differentiation, we express the derivative of $h(\epsilon)$ via partial derivatives as

[TABLE]

Letting $\epsilon=0$, we find

[TABLE]

*We substitute $\lambda^{\prime}(0)=\bm{v}^{(i)})^{T}\Delta{L}\bm{v}^{(i)}$ from Eq. (16) to obtain Eq. (18). *

Appendix C Proof of Corollary 3.19

Proof C.1**.**

For unweighted networks, all non-diagonal entries $L_{ij}$ in are either 0 (if there is no edge) or -1 (if there is an edge $(i,j)\in\mathcal{E}$). The addition of an edge $(p,q)$ implies $L_{pq}=L_{qp}=-1$, and because $\sum_{i}L_{ij}=0$ by definition, $L_{ii}=\sum_{j\not=i}L_{ij}$ and any perturbation of off-diagonal elements must be reflected in the diagonal elements. Consideration of an edge removal leads to an analogous result, albeit with an opposite sign, and therefore $\Delta L$ must be of the form given by Eq. (22). It is straightforward to show

[TABLE]

*We substitute this result into Eq. (18) to obtain Eq. (23). *

Appendix D Proof of Theorem 3.23

Proof D.1**.**

The process of randomly rewiring an edge $(p,q)$ to $(r,s)$ can be decomposed into two steps. The first step is removing an edge $(p,q)$ from the original graph $G^{(0)}$, resulting in an intermediate graph $G^{(1)}$. The second step is adding an edge $(r,s)$ to the graph $G^{(1)}$, resulting in the rewired graph $G^{(2)}$. Let $L^{(0)}$ denote the Laplacian matrix of the original graph $G^{(0)}$, $L^{(1)}$ denote the Laplacian matrix of the intermediate graph $G^{(1)}$, and $L^{(2)}$ denote the Laplacian matrix of the rewired graph $G^{(2)}$, then we have $L^{(1)}=L^{(0)}+\Delta L^{(0)}$, $L^{(2)}=L^{(1)}+\Delta L^{(1)}$. In terms of our previous notations, we have

[TABLE]

D.1 Removing an edge

Since removing an edge $(p,q)$ means $A_{pq}$ and $A_{qp}$ change from $1$ to [math], the elements of $\Delta L^{(0)}_{ij}$ are given by

[TABLE]

Using that edges are removed uniformly at random, the expected values of $\{\Delta L^{(0)}_{ij}\}$ are given by

[TABLE]

Since there are $M$ edges in total, and we can only remove an edge when $A_{ij}=A_{ji}=1$, we can write down the probabilities as

[TABLE]

and

[TABLE]

We substitute these probabilities into Eq. (45) to obtain

[TABLE]

D.2 Adding an edge

Since adding an edge $(r,s)$ means $A_{rs}$ and $A_{sr}$ change from [math] to $1$, the elements $\{\Delta L^{(1)}_{ij}\}$ are given by

[TABLE]

which have the expectations

[TABLE]

Since there are $\frac{N(N-1)}{2}$ possible edges in total for a graph with $N$ nodes, and we can only add an edge when $A_{ij}=A_{ji}=0$, and $\{i,j\}\neq\{p,q\}$. Therefore, there are $R=\frac{N(N-1)}{2}-M+1$ possible edges to add, yielding the probabilities

[TABLE]

and

[TABLE]

We substitute these probabilities into Eq. (50) to obtain

[TABLE]

D.3 Rewiring an edge

By the linearity of expectation, we have

[TABLE]

We substitute Eqs. 48 and 53 into 54 to obtain

[TABLE]