Designing Optimal Multiplex Networks for Certain Laplacian Spectral Properties

TL;DR

This paper investigates how to optimally design interlayer edges in multiplex networks to enhance spectral properties like algebraic connectivity and spectral radius within budget constraints, revealing phase transitions in optimal configurations.

Contribution

It introduces a dual formulation linking spectral optimization problems to graph embedding, providing new insights into optimal multiplex network design under budget limitations.

Findings

Optimal interlayer weights are uniform below a budget threshold for algebraic connectivity.

Embedding dimension changes at certain budget thresholds, reflecting phase transitions.

Optimal configurations shift from one-dimensional to multi-dimensional embeddings as budget increases.

Abstract

We discuss the design of interlayer edges in a multiplex network, under a limited budget, with the goal of improving its overall performance. We analyze the following three problems separately; first, we maximize the smallest nonzero eigenvalue, also known as the algebraic connectivity; secondly, we minimize the largest eigenvalue, also known as the spectral radius; and finally, we minimize the spectral width. Maximizing the algebraic connectivity requires identical weights on the interlayer edges for budgets less than a threshold value. However, for larger budgets, the optimal weights are generally non-uniform. The dual formulation transforms the problem into a graph realization (embedding) problem that allows us to give a fuller picture. Namely, before the threshold budget, the optimal realization is one-dimensional with nodes in the same layer embedded to a single point; while, beyond the threshold, the optimal embeddings generally unfold into spaces with dimension bounded by the multiplicity of the algebraic connectivity. Finally, for extremely large budgets the embeddings revert again to lower dimensions. Minimizing the largest eigenvalue is driven by the spectral radius of the individual networks and its corresponding eigenvector. Before a threshold, the total budget is distributed among interlayer edges corresponding to the nodal lines of this eigenvector, and the optimal largest eigenvalue of the Laplacian remains constant. For larger budgets, the weight distribution tends to be almost uniform. In the dual picture, the optimal graph embedding is one-dimensional and non-homogeneous at first and beyond this threshold, the optimal embedding expands to be multi-dimensional, and for larger values of the budget, the two layers fill the embedding space. Finally, we show how these two problems are connected to minimizing the spectral width.

Equations

$$L(w):=\sum _{\{i,j\}\in E_1\cup E_2}{L}_{ij}+\sum _{\{i,j\}\in E_3}{w}_{ij}{L}_{ij},$$

$$L(w)=\left[\begin{array}{cc}{L}_1+W& -W\\ -W& L_2+W\end{array}\right]$$

$$\underset{w_{ij},{\lambda }_2,\mu }{\mathrm{maximize}}$$

$$\sum _{ij\in E_3}{w}_{ij}{L}_{ij}+L_0+\mu \mathbf{e}{\mathbf{e}}^T-{\lambda }_{2}I⪰0$$

$$\sum _{ij\in E_3}{w}_{ij}=c$$

$$w_{ij}\ge 0$$

$$c^{\ast }=\frac{n}{2}{\lambda }_2\left[{\left(L_1^{†}+L_2^{†}\right)}^{†}\right]$$

$${\lambda }_2\left[L\left(w^{\ast }\right)\right]\le {\lambda }_2(L_{ave})$$

$$\underset{\xi \in \mathbb{R},X\in {\mathbb{R}}^{n\times n}}{\mathrm{maximize}}$$

$$⟨X,I⟩=1$$

$$⟨X,e{e}^T⟩=0$$

$$⟨X,L_{ij}⟩\le -\xi \forall \{i,j\}\in E_3$$

$$X⪰0$$

$$\underset{\xi \in \mathbb{R},u_i\in {\mathbb{R}}^n}{\mathrm{maximize}}$$

$$\sum _{i\in V}\Vert u_i{\Vert }^2=1$$

$$\sum _{i\in V}{u}_i=0$$

$$\Vert u_i-u_j{\Vert }^2\le -\xi \forall \{i,j\}\in E_3$$

$$\underset{{\hat{w}}_{i,j}\in {\mathbb{R}}^{E_3}}{\mathrm{minimize}}$$

$$c\sum _{i,j\in E_3}{\hat{w}}_{ij}{L}_{ij}+(\sum _{\{i,j\}\in E_3}{\hat{w}}_{ij})L_0$$

$$+\hat{\mu }\mathbf{e}{\mathbf{e}}^T-I⪰0$$

$${\hat{w}}_{ij}\ge 0\forall \{i,j\}\in E_3$$

$$\underset{{\hat{u}}_i\in {\mathbb{R}}^n}{\mathrm{maximize}}$$

$$c\Vert {\hat{u}}_i-{\hat{u}}_j{\Vert }^2+\sum _{\{k,l\}\in E_1}\Vert {\hat{u}}_k-{\hat{u}}_l{\Vert }^2+$$

$$\sum _{\{k,l\}\in E_2}\Vert {\hat{u}}_k-{\hat{u}}_l{\Vert }^2\le 1\forall \{i,j\}\in E_3$$

$$\sum _{i\in V}{\hat{u}}_i=0$$

$${\hat{u}}_i^{\ast }=\{\begin{array}{ll}h& \text{if}i\in V_1\\ -h,& \text{if}i\in V_2\end{array}$$

$$\mathcal{U}=-\frac{1}{2}\sum _{i<j\in V_1\cup V_2}\Vert {\hat{u}}_i-{\hat{u}}_j{\Vert }^2.$$

$$\sum _{j\sim i}{T}_{ij}\frac{{\hat{u}}_i-{\hat{u}}_j}{\Vert {\hat{u}}_i-{\hat{u}}_j\Vert }={\hat{u}}_i\forall i\in V$$

$$T_{ik}\frac{{\hat{u}}_i-{\hat{u}}_k}{\Vert {\hat{u}}_i-{\hat{u}}_j\Vert }+\sum _{\begin{array}{c}j\sim i\\ j\in G_1\end{array}}{T}_{ij}\frac{{\hat{u}}_i-{\hat{u}}_j}{\Vert {\hat{u}}_i-{\hat{u}}_j\Vert }={\hat{u}}_i,$$

$$\sum _{j\in G_1}{a}_{ij}{T}_{ij}\frac{{\hat{u}}_i-{\hat{u}}_j}{\Vert {\hat{u}}_i-{\hat{u}}_j\Vert }=0$$

$$\underset{w_{ij}}{\mathrm{minimize}}$$

$$\sum _{\{i,j\}\in E_3}{w}_{ij}{E}_{ij}+L_0-{\lambda }_{n}I⪯0$$

$$\sum _{\{i,j\}\in E_3}{w}_{ij}\ge c$$

$$w_{ij}\ge 0$$

$${\lambda }_n[L(w)]=\underset{\Vert v\Vert \ne 0}{\max }\frac{v^{T}L(w)v}{\Vert v{\Vert }^2}$$

$${\lambda }_n^{\ast }(c):=\underset{\underset{w^{T}1=c}{w\ge 0}}{\min }{\lambda }_n[L(w)]$$

$${\lambda }_n^{\ast }(c)\equiv {\lambda }_N^1\text{for 0≤c≤c1∗.}$$

$$W{v}_N^1=0$$

$${\lambda }_n^{\ast }(c)\ge {\lambda }_N^1\forall c\ge 0.$$

$${\lambda }_{n-1}^{\ast }(c)<{\lambda }_N^1.$$

Full text

††preprint: APS/123-QED

I Introduction

Multiplex networks consist of distinct layers interacting together in diverse social, economical, transportation, and biological networks (Kim and Goh, 2013; Kivelä et al., 2014). Coupling structure of network layers is shown to affect connectivity and robustness properties of the entire system (Lee et al., 2012). With the existing literature mostly limited to single networks, further systematic work is needed to discover the underlying mechanism in multiplex networks (Kivelä et al., 2014).

Let $G=\left(V,E\right)$ represent an undirected network and by $V=\left\{1,\ldots,n\right\}$ and $E\subset{V\choose 2}$, we denote the set of nodes and links. For a link $e$ between nodes $i$ and $j$, i.e., $e:\{i,j\}\in E$, we define a nonnegative value $w_{ij}$ as the weight of the link. Given $G$ a multiplex network, let $G_{1}=\left\{V_{1},E_{1}\right\}$ and $G_{2}=\left\{V_{2},E_{2}\right\}$, $|V_{1}|=|V_{2}|$ represent the layers, and a bipartite graph $G_{3}=\left\{V,E_{3}\right\}$ with $E_{3}\subseteq\{\{v_{1},v_{2}\}:v_{i}\in V_{i}\}$ are connecting the layers (Fig. 1). Throughout the paper, we use the term intralayer links for $E_{1}$ and $E_{2}$, and interlayer links for $E_{3}$.

The links in $G_{3}$ bridge $G_{1}$ and $G_{2}$ and should be chosen strategically, for instance in a way that minimizes the disruption of the flow of information, electric power or goods, or to avoid failures against attackers and possible errors that can fragment the system or cause cascading phenomena (Buldyrev et al., 2010). The edge weights of $G_{3}$ are the sole design parameters.

The Laplacian matrix is defined as

[TABLE]

where $L_{ij}:=(\delta_{i}-\delta_{j})(\delta_{i}-\delta_{j})^{T}$, for each link $\{i,j\}$, and $\delta_{i}$ is the delta function at vertex $i$. In particular, we think of the Laplacian matrix of $G$ as a function of the interlayer weights $w$. Enumerating the vertices in $V_{1}$ followed by the vertices in $V_{2}$, we can write $L(w)$ in block form in terms of the Laplacian matrices of the layers, $L_{1}$ and $L_{2}$, as follows:

[TABLE]

In particular, we will assume from now on that $W=\text{diag}(w)$, meaning that $E_{3}$ consists of a perfect matching, see Fig. 1.

Remark 1**.**

All eigenvalues of $L(w)$ are nondecreasing functions of $w$, because (2) can be thought of as perturbing $L(w)$ using a positive semidefinite matrix.

Our goal is to allocate weights on the interlayer links, subject to a budget such that $\sum w_{ij}=c$, to obtain extremal spectral properties. More specifically, we are interested in maximizing the smallest positive eigenvalue of the Laplacian, minimizing the largest eigenvalue, and minimizing the difference between the two. After formulating the primal-dual program and deriving its properties, the equivalent graph realization problem in each case is extracted and its features with respect to the multiplex network’s structure are identified.

Recall that the Laplacian matrix $L(\omega)$ is positive semidefinite and has (at least for connected networks) one zero eigenvalue with eigenvector $\bm{e}=[1,\dots,1]^{T}$, the vector of all ones of appropriate length. The eigenvalues of $L(\omega)$ are ordered as $0=\lambda_{1}(\omega)\leq\lambda_{2}(\omega)\leq\lambda_{3}(\omega)\leq\dots\leq\lambda_{n}(\omega)$.

The main contributions of the paper are traceable under several circumstances. While the literature is limited to uniformly weighted interconnections, we study optimal distributions (not necessarily uniform) of interlayer weights and define three types of optimization criteria based on our inter-structure design objectives. In addition to maximizing algebraic connectivity $\lambda_{2}(\omega)$, we consider the problem of minimizing spectral radius $\lambda_{n}(\omega)$ and finally minimizing the spectral width $\lambda_{n}(\omega)-\lambda_{2}(\omega)$ with motivating their numerous applications. Most of our results for all three problems are analytical so that there is no need for resorting to numerical solutions in such conditions. However, formulating the considered problems in a convex framework enables efficient numerical approaches in regions where there is no analytical solution yet. Moreover, we provide a primal-dual setting that reflects a multi-sided view of each problem; i.e. the geometric dual formulation reduces the primal problems into graph embedding problems related to several physical aspects that also enables easier interpretations.

I.1 Algebraic connectivity $\lambda_{2}$

From now on we assume the network to be connected, thus the second smallest eigenvalue is positive in this case, and its magnitude reflects the level of connectedness of the graph. In fact, $\lambda_{2}$ is called the algebraic connectivity of $G$; Fiedler (1973) showed that algebraic connectivity increases monotonically by adding links. The algebraic connectivity can also be considered as a measure of network robustness (Jamakovic and Uhlig, 2007). Moreover, various bounds in graph partitioning, optimal graph labeling, min-sum problems, or bandwidth optimization can be obtained using $\lambda_{2}$ as a determining factor (Juvan and Mohar, 1993; Helmberg et al., 1995). The convergence speed of various processes such as mixing Markov chains on graphs (Brémaud, 2013), reaching consensus in multi-agent systems (Jadbabaie et al., 2003; Olfati-Saber and Murray, 2004; Olfati-Saber, 2006; Tanner et al., 2007), or diffusion dynamics on networks (Gomez et al., 2013) are controlled by the second smallest eigenvalue of the Laplacian.

In our multiplex model with varying weights $w$, the algebraic connectivity for constant $w$ grows linearly with $w$ up to a critical $w^{*}$, and then has a nonlinear behavior afterwards Gomez et al. (2013). Bounds for $w^{*}$ are found in Radicchi and Arenas (2013) and its exact value is found in Darabi Sahneh et al. (2015). In (Martín-Hernández et al., 2014), the structure of the interlayer links $G_{3}$ is studied by comparing two configurations: diagonal (one-to-one) vs. random.

In a single layer graph $G$, with variable edge-weights subject to a total budget, Boyd et al. (2004) and Goring et al. (2008) discuss how maximizing $\lambda_{2}$ corresponds to a dual semidefinite optimization problem and show that the optimal solutions of the dual are related to the eigenvectors of the optimal algebraic connectivity. It turns out that the dual may be interpreted as an embedding of the single-layer graph in $\mathbb{R}^{n}$ (optimal realization of the graph in Euclidean space), and the optimal embedding has structural properties tightly connected to the separators of the graph.

In this paper we consider the setting of multi-layer networks with a one-to-one interconnected structure and maximize the algebraic connectivity of the whole given a limited total budget. We address this by formulating and studying the properties of primal and dual problems. Shakeri et al. (2016) show that strong duality holds and revisit the fact that, up to a threshold budget, the solution to the primal problem is the uniform distribution with identical weights. For larger budgets, the optimal weights are generally not uniform. The dual formulation transforms the problem into a graph realization (embedding) problem. We show that before the threshold budget, the optimal realization is one-dimensional and consists of nodes in the same layer clumped together; while, beyond the threshold, the eigenvalues of the Laplacian coalesce and optimal embeddings generally take place in spaces with more than one dimension. Finally, for very large budgets the embedding is again one-dimensional.

I.2 Laplacian spectral radius $\lambda_{n}$

The largest eigenvalue $\lambda_{n}$ provides useful information about graph structural properties (see Reith (2012) and references therein), various bounds for algebraic properties in graphs (Juvan and Mohar, 1993; Helmberg et al., 1995), and applications in game theory (Li and Cao, 2009). The largest eigenvalue must be sufficiently small for stability of formation control algorithms when the agent dynamics are prone to high-gain instability or unmodeled dynamics (Bai and Arcak, 2010). Optimization of the largest eigenvalue is related to finding a weighted tree with the largest spectral radius of the Laplacian matrix (Tan, 2010; Li and Tian, 2011). Fiedler (1990) considered the problem of minimizing the maximum eigenvalue of the weighted Laplacian for trees and bipartite graphs and investigated connections with doubly stochastic matrices. Goring et al. (2012) studied the problem of minimizing the maximum eigenvalue of the Laplacian of single-layer graphs, and by transforming the corresponding dual problem studied the graph realization problem.

I.3 Spectral width $\lambda_{n}-\lambda_{2}$

The difference between the second smallest and largest eigenvalue, or the spectral width, provides important information about several bounds on different graph metrics. For instance, the value of a uniform sparsest cut falls in the interval bounded by the second smallest and the largest eigenvalue of the Laplacian divided by the number of nodes (see Beineke et al. (2004, Lemma 4.1)). Moreover, a small spectral width implies that the graph is Hamiltonian, i.e. a graph possessing a cycle visiting each vertex exactly once (Butler and Chung, 2010). Also instability can occur in cooperative motion control when the frequency of the parametric perturbation is near an eigenvalue of the Laplacian (Bai and Arcak, 2010). Thus, minimizing the spectral width will shrink the frequency band of parametric resonance. Goring et al. (2013) studied the problem of minimizing the spectral width and discovered connections between this hybrid problem and the separate problems of maximizing the algebraic connectivity and minimizing the spectral radius. In this paper, the equivalent problem for multiplex networks is considered by deriving and studying the primal and dual embedding problems. In particular, we show that the primal and embedding problems corresponding to spectral width minimization can be related to the individual problems of maximizing $\lambda_{2}$ and minimizing $\lambda_{n}$.

I.4 Graph realization problem

Inspired by Sun et al. (2006), we reformulate the dual problem as a graph realization problem and study the connections to the graph’s structure. Since the Laplacian is a positive semidefinite matrix, the solution of the dual program can be represented by a Gram matrix. The dual problem is seen to be equivalent to a realization of the graph in $\mathbb{R}^{n}$, by assigning the graph nodes to the vectors of the Gram representation. It is of particular interest to find the smallest dimension for which a graph is realizable. Graph realizations related to extremal Laplacian eigenvalues have close connections with problems in other areas. For example, graph realization problems arise in the determination of molecular conformation (Hendrickson, 1995). An interesting problem also appears in manifold learning where the structure of a low dimensional manifold is constructed from sampled high dimensional data (Weinberger et al., 2004; Weinberger and Saul, 2006).

In Sections II, III, and IV, we formulate maximizing the second smallest eigenvalue, minimizing the maximum eigenvalue, and minimizing the difference between the second smallest and the largest eigenvalue in multiplex networks, respectively. We inspect and demonstrate the properties of primal-dual and embedding problems in each case. Section V is devoted to concluding remarks.

II Maximizing $\lambda_{2}$

This section examines properties of the graph resulting from maximizing the second smallest eigenvalue under a budget constraint on the interlayer edge weights. The original work for single-layer graphs was proposed by Fiedler (1989) who maximized the graph connectivity under the budget constraint that the total of the edge weights equals the number of edges. Shakeri et al. (2016) follow Goring et al. (2008, 2011) and propose the following formulation for multi-layer networks

[TABLE]

where $L_{0}=\sum_{ij\in E_{1}\cup E_{2}}L_{ij}$ is the Laplacian for the disjoint union of the layers. In the semidefinite constraint, the free variable $\mu$ serves to shift the zero eigenvalue, and ensure that $\lambda_{2}$ becomes the smallest eigenvalue of $\sum_{ij\in E_{3}}w_{ij}L_{ij}+L_{0}+\mu\bm{e}\bm{e}^{T}$. In this regard, when the optimal solution $(w_{ij}^{*},\lambda_{2}^{*},\mu^{*})$ is attained, $\lambda_{2}^{*}$ is the smallest eigenvalue of $\sum_{ij\in E_{3}}w_{ij}^{*}L_{ij}+L_{0}+\mu^{*}\bm{e}\bm{e}^{T}$ or equivalently the second smallest eigenvalue of $\sum_{ij\in E_{3}}w_{ij}^{*}L_{ij}+L_{0}$.

Shakeri et al. (2016) show that, for the special case of multiplex networks with identical layers the uniform weight distribution is always optimal for (3). Moreover, the corresponding optimal algebraic connectivity increases linearly with the budget $c$ up to a threshold $c^{*}>0$ and then remains constant after that. When the layers are not identical, the uniform distribution is optimal only for budgets $c$ up to the threshold $c^{*}>0$, and increasing the budget past the threshold, yields nonuniform optimal weight distributions. Therefore, in this case, it is still possible to improve the algebraic connectivity by increasing the budget beyond $c^{*}$. Figure 2 shows uniform and nonuniform optimal weight distributions in two budget regimes for a small multiplex.

More specifically, before the threshold, $\underset{\sum{w_{ij}}=c}{\text{max}}\lambda_{2}(w_{ij})=\frac{4c}{n}$, $\forall c\leq c^{*}$, and beyond the threshold it remains under $4c/n$ for all $c>c^{*}$. Lacking the knowledge of these regimes casts serious challenges on proposing heauristics similar to Ghosh and Boyd (2006).

The threshold budget is calculated in (Darabi Sahneh et al., 2015) (and in Shakeri et al. (2016) with a different proof) to be

[TABLE]

where † denotes the Moore-Penrose pseudoinverse. An upper-bound for the algebraic connectivity is also given in Radicchi and Arenas (2013)

[TABLE]

where $L_{ave}=\frac{L_{1}+L_{2}}{2}$. Figure 3 compares the optimal value of algebraic connectivity to the one obtained by the uniform distribution as the budget $c$ varies past the threshold, for four different random layer structures (see SM .9 for network models). In all cases, the optimal distribution gives a higher algebraic connectivity after the threshold. We also observe that for $c>c^{*}$, $\lambda_{2}$ increases at a slower rate than before the threshold.

In Figure 4 , we consider two Erdős-Rényi layers and vary the difference of the algebraic connectivity of the layers, i.e. we look at small and large values of $\left|\lambda_{2}(L_{1})-\lambda_{2}(L_{2})\right|$. In the case when the layers have similar algebraic connectivities, the threshold $c^{*}$ has a larger value. This enlarges the linear part of the diagram, thus postponing the nonlinear region and its slow growth. As a result, nodes in multiplex graphs with individual layers sharing near connectivity properties are synchronized more easily (with less budget) and superdiffusion occurs (Gomez et al., 2013). To achieve the same degree of interdependent connectivity when layers have much different connectivity properties, a larger budget is required. In the extreme case of identical layers, the upperbound (5) is achieved at the threshold. Similar behaviors are observed for the other network models.

II.1 Dual and embedding problems for $\lambda_{2}$

The dual problem for (3) is obtained by the Lagrangian approach (Boyd and Vandenberghe, 2004), (see SM .1), and can be written as follow:

[TABLE]

where $\langle X,L_{0}\rangle=\operatorname{Tr}(L_{0}^{T}X)=\sum_{\{i,j\}\in E_{1}\cup E_{2}}x_{ii}+x_{jj}-2x_{ij}$.

Proposition 2**.**

The feasible set of the dual problem is not empty.

Proof.

See SM .2. ∎

Proposition 3**.**

Strong duality holds for the primal and dual problems (3) and (6), and the dual problem attains its optimal solution.

Proof.

See SM .3. ∎

Since $X$ is a positive semi-definite matrix, we can use its Gram representation $X=U^{T}U$, where $U\in\mathbb{R}^{n\times n}$ (Sun et al., 2006), and rewrite (6) as:

[TABLE]

Proposition 4**.**

The projections of optimal embedding onto one-dimensional subspaces yield eigenvectors for the algebraic connectivity.

Proof.

See SM .4.∎

For connected single-layer networks, Goring et al. (2008) scale the weights by $c\lambda_{2}\neq 0$, i.e., they consider $\frac{w_{ij}}{c\lambda_{2}}=\hat{w}_{ij}$, where $c$ is the budget. We can use the same procedure here, and obtain a scaled version of the primal-dual problem for multiplex networks. The scaled primal problem of (6) is

[TABLE]

and the scaled dual (embedding) problem (7) is written as (see SM .1)

[TABLE]

It is known that there are transformations that map optimal solutions of the primal and dual problems to the scaled ones (Reith, 2012). Together with Proposition 3, this shows that strong duality holds also for the scaled problem.

Proposition 5**.**

For budget values up to the threshold $c^{*}$, the optimal solution of the embedding problem (9) is given as

[TABLE]

where $\bm{h}=\langle h\rangle$ is a one-dimensional subspace.

Proof.

See SM .5. ∎

The embedding (10) implies each layer clumps together at opposite ends, in the optimal realization (Figure 5). In this case, the Fiedler cut distinguishes the individual layers (Van Mieghem, 2010).

Remark 6**.**

The algebraic multiplicity of $\lambda_{2}(L)$ sets an upper-bound on the dimension of realization (Helmberg and Reiss, 2010). This can be understood from Proposition 4 by recalling that the dimension of the eigenspace corresponding to $\lambda_{2}$ is at most the multiplicity of $\lambda_{2}$.

II.2 Interpretation of the embedding problem

Here we expand on the interpretation of problem (9). The goal is to maximize the spread (variance) of the vectors $\hat{u}_{i}$ subject to a constraint involving the neighbor relations (and a fixed barycenter).

Note that the budget $c$ appears uniquely in the inequality constraint. For small values of $c$, this inequality puts less limitations on the embedding distances between connected nodes via the interlayer links than the total embedding distances between nodes within the layers. As the budget increases, the interlayer distances pay a higher and higher toll and the intralayer ones gain more freedom (we say that they “unfold”). In particular, the layer with lower algebraic connectivity enjoys more flexibility and will unfold faster.

Another interpretation of problem (9) consists of thinking of the vectors $\hat{u}_{i}$ as the positions of some unit masses subject to repelling and attracting forces.

Here we follow the mechanical interpretation given in Sun et al. (2006, Sec. 4.6) in the case of one layer. Similarly, we define a potential energy $\mathcal{U}$ for the $n$-point,

[TABLE]

By rotation and translation invariance, minimizing $\mathcal{U}$ is seen to be equivalent to maximizing the objective function in (9). The constraints in (9) can be seen as bounds on the elastic potential energy along the edges.

The total force on each node can be found by differentiating this energy function and the resultant force is toward the origin with a magnitude proportional to the distance of the point. Consequently, we can write the static equilibrium condition as

[TABLE]

where $T_{ij}$ is the tension in the imaginary link (similar to a spring) between the two nodes $i$ and $j$. Since the interlayer connections are one to one, assume without loss of generality that $i\in G_{1}$, and $k\in G_{2}$ is the matched neighbor of $i$ in the other layer. Then (11) becomes

[TABLE]

Therefore, at each node, the sum of forces due to interlayer and intralayer springs is balanced by a repulsive force that is proportional to the distance of the node to the origin. We can use this to explain the embedding configurations, by combining the inequality constraints with the orientation of the force field. For example, by Proposition 5, for budgets $c<c^{*}$ below the threshold, we have

[TABLE]

hence $T_{ik}=w_{ik}$, in this case.

In Figure 6, we consider two layers (of Watts-Strogatz type) whose algebraic connectivities are very close. In Figure 6a, we plot the behavior of $\lambda_{2}$ for the resulting multiplex, and observe three different regimes as the budget increases. For $c<c^{*}$, $\lambda_{2}$ is simple and grows linearly, after the threshold, the multiplicity increases to two. Finally, there is another phase shift $c^{**}>c^{*}$, so that for large budgets $\lambda_{2}$ becomes simple once again.

Recall that by Remark 6, the multiplicity of $\lambda_{2}$ provides an upperbound for the the embedding dimension in each regime. This can be seen is in Figures 6b-6f shows. For low budgets the embedding looks like Figure 5, i.e., the nodes are clumped in each layer. In the second regime, when $c^{*}<c<c^{**}$, due to the nonuniform interlayer forces interacting with intralayer forces, the embedding unfolds two-dimensionally, in Figures 6b and 6c. Notice that the different layers are still distinguishable in two-dimensional embeddings for budgets just above the threshold $c^{*}$. By further increasing $c$ towards $c^{**}$, the stiffened interlayer springs draw the two layers towards each other, thus decreasing the distance between them (see 6d and Figure 6e); and eventually, after $c^{**}$, the two layers combine and operate as a whole one-dimensional embedding in Figure 6f.

The difference between the one-dimensional embeddings in Figures 6a and 6f is that, for small budgets each layer collapses to a pair of distinct points, while for large budgets, each matched pair from different layers collapses to a point.

In Figure 7, we consider two layers with very different algebraic connectivity. We observe that the layer with weaker connectivity unfolds sooner. Notice also, in this case the embedding becomes three dimensional.

The orientation of the embeddings discussed above bring forward an optimal criterion on the structure of the embeddings known as separator-shadow established by Goring et al. (2008). The result is that, the structural properties of optimal embeddings are governed by the separators of the graph. Let separator $S$ split the graph into at least two separate components between which no edges exist (the union of $S$ and the components constitute the whole graph). Then, all but one component have the property that, the straight line segment between the origin and each node of the component intersects the convex hull of $S$–reminiscent of the forces that are aligned toward the origin. In other words, if we look at origin as a light source, nodes of all but one separated component is embedded in the shadow of the convex hull of the separator. We investigate this in multiplex networks. First, let $G$ be connected, and $w_{ij}\geq 0$ be a feasible solution of (8), i.e. it embodies the inequality constraints and let $G_{w}=\left(V,E_{w}:=\{ij\in E:w_{ij}>0\}\right)$. The embedding topology follows the seperator-shadow theorem stated in Goring et al. (2008) (see SM .6).

An important feature of geometric dual formulation in this paper is that, by Proposition 4, different embedding patterns are informative of different optimal diffusion phases. Particularly, Proposition 5 shows that for $c\leq c^{*}$, the embedding is one dimensional and nodes in the same layer are clumped; consequently, the interlayer edges are not strong enough to unfold the embedding structures. In such conditions, diffusion within each layer is through intralinks, and interlinks do not contribute much to diffusion between nodes in the same layer. For $c>c^{*}$, we observe in Figure 6 that stronger interlayer links start to unfold the layers and with increasing $c$, the distances between nodes in the same layer increase and thus interconnected nodes in different layers approach each other. This is indicative of transition from the intralayer phase of diffusion to the interlayer phase. Furthermore, in some circumstances, for instance, when one individual network has much larger algebraic connectivity than the other one, increasing the budget past $c^{*}$ unfolds the layer with smaller algebraic connectivity. We observe an obvious example of this situation in Figures 7b and 7c. This is due to higher robustness level of the layer with larger algebraic connectivity, and is indicative of interlayer optimal diffusion mode in the more weakly connected layer, versus intralayer optimal diffusion mode in the more connected one.

III Minimizing $\lambda_{n}$

In this section, we study the problem of minimizing the largest eigenvalue, $\lambda_{n}$, of the Laplacian in multiplex networks. For the total cost $c$ of interlayer links, the primal problem is defined as

[TABLE]

To get some insight into the solution of the primal problem (13), consider the characterization of the largest eigenvalue in terms of Rayleigh quotients, as

[TABLE]

The optimal weight problem (13) is then

[TABLE]

for a given budget $c\geqslant 0$. Since $L$ is an affine function of $w$, and $\lambda_{n}$ is a convex function of $L$, it follows that (15) is a convex optimization problem. Indeed, (15) can be recast as the semidefinite programming (13) and may be solved using standard numerical methods.

In Solé-Ribalta et al. (2013), the weight distribution is assumed to be uniform. In that case, for small values of $c$, $\lambda_{n}$ grows approximately linearly with $c$, namely, $\lambda_{n}=\lambda_{N}^{0}+2c/N$, where $\lambda_{N}^{0}:=\text{max}(\lambda_{max}(L_{1}),\lambda_{max}(L_{2}))$, and $N$ is the number of nodes in each layer.

Here we will show that the optimal $\lambda_{n}^{*}$ in (13), may actually be constant for small budgets.

Theorem 7**.**

Assume that $\lambda_{max}(L_{1})>\lambda_{max}(L_{2})$ and that $\lambda_{N}^{1}:=\lambda_{max}(L_{1})$ is simple for $L_{1}$ with eigenvector $v_{N}^{1}$. Suppose that the nodal set $\{x\in V(G_{1}):v_{N}^{1}(x)=0\}$ is non-empty, and define $c_{1}^{*}>0$ to be the largest budget such that for $0\leq c<c_{1}^{*}$, the optimal $\lambda_{n}^{*}(c)$ is simple. Then,

[TABLE]

Moreover, in this case, a weight $w$ with $w^{T}\bm{1}=c$ is optimal for (13) if and only if

[TABLE]

where $W=\operatorname{diag}(w)$ and [math] is a zero vector.

Proof.

First note that, by Remark (1), the optimal largest eigenvalue $\lambda_{n}^{*}$ for (13) is a nondecreasing function of $c$. So

[TABLE]

By Weyl’s Theorem (see Roger and Charles (1994, Chapter 3)), since $L(w)$ is a small perturbation of $L(0)$, there is $c_{1}>0$, so that for $0\leq c<c_{1}$,

the corresponding $\lambda_{n}^{*}(c)$ is also simple; 2. 2.

and the second largest eigenvalue $\lambda_{n-1}^{*}(c)$ satisfies

[TABLE]

Note that $c_{1}\leq c_{1}^{*}$, because condition 2. is in principle adding an extra restriction.

Fix a budget $0\leq c<c_{1}$, and let $w$ be a feasible weight with $w^{T}\bm{1}=c$. Assume first that $w$ satisfies (17). Then

[TABLE]

Therefore, $\lambda_{N}^{1}$ is an eigenvalue of $L(w)$ with corresponding eigenvector $v:=\left({v_{N}^{1}}^{T},0\right)^{T}$, and by (19), it must be the largest eigenvalue of $L(w)$. In particular, by (18), $w$ is optimal and $\lambda_{n}^{*}(c)=\lambda_{N}^{1}$.

This shows that (17) is sufficient for optimality of $w$. In addition, since the nodal set is non-empty, there are many weights that satisfy (17), indeed any weight supported on the nodal set will be optimal. So we have also established (16) for $0\leq c\leq c_{1}$, by continuity up to $c_{1}$. To conclude, note that for $c=c_{1}$, either $\lambda_{n}^{*}(c_{1})$ is not simple, or $\lambda_{n-1}^{*}(c_{1})=\lambda_{N}^{1}$, in which case $\lambda_{n-1}^{*}(c_{1})=\lambda_{n}^{*}(c_{1})$, and thus $\lambda_{n}^{*}(c_{1})$ is again not simple. This shows that $c_{1}=c_{1}^{*}$.

The only direction left, is to show that (17) is also necessary for optimality in this case.

So, assume that $w$ is optimal. Using the layer structure, the vector $v$ in (14) can be written as $v=(v_{1}^{T},v_{2}^{T})^{T}$, where $v_{i}$ is the restriction of $v$ to layer $i$. Then, (14) for $\lambda_{n}[L(w)]=\lambda_{N}^{1}$ implies that

[TABLE]

If we decompose $v_{1}$ into two orthogonal components as $v_{1}=\alpha v_{N}^{1}+u_{1}$, for a scalar $\alpha$, where $u_{1}^{T}v_{N}^{1}=0$, then $\|v_{1}\|^{2}=\alpha^{2}+\|u_{1}\|^{2}$ and

[TABLE]

Since (22) holds for every $\alpha$, we must have $(v_{N}^{1})^{T}Wv_{N}^{1}=0$. This concludes the ‘only if’ direction. ∎

Proposition 8**.**

At the threshold $c_{1}^{*}$, the matrix $Q+2W^{\star}$ has a zero eigenvalue, where $Q=\bar{L}-\tilde{L}\bar{L}^{\dagger}\tilde{L}$ and $\bar{L}=\frac{L_{1}+L_{2}}{2}-\lambda_{N}^{1}I$, $\tilde{L}=\frac{L_{1}-L_{2}}{2}$.

Proof.

Using the approach in Darabi Sahneh et al. (2015), we write the eigenvalue problem $Lv=\lambda v$ as

[TABLE]

where $v=[v_{1}^{T},v_{2}^{T}]^{T}\in\mathbb{R}^{2N}$ satisfies the following eigenvector normalization:

[TABLE]

Differentiating (23) and (24) with respect to $c$ yields the governing equations for the eigenderivatives $\frac{dv_{1}}{dc}$, $\frac{dv_{2}}{dc}$, and $\frac{d\lambda}{dc}$

[TABLE]

For $0\leq c<c_{1}^{*}$, $\lambda=\lambda_{N}^{1}$, $v_{1}=v_{N}^{1}$, $v_{2}=0$, therefore

[TABLE]

Consider

[TABLE]

The matrix $L$ can have repeated eigenvalue only if $M$ is singular. Hence, at the threshold $c_{1}$ there exists $x\neq 0$ such that $M(W^{*})x=0$, thus

[TABLE]

Using Theorem 7 and (17), the inner product of the first row on both sides of (27) with $v_{N}^{1}$ shows that $x$ is of the form $x=\left[x_{1}^{T}\ \ x_{2}^{T}\ \ 0\right]^{T}$. Then, a linear transformation $y_{1}=x_{1}-x_{2}$, $y_{2}=x_{1}+x_{2}$, in (27) implies that

[TABLE]

Eliminating $y_{2}$ in (28) yields

[TABLE]

where $\bar{L}^{\dagger}$ is the Moore Penrose pseudo-inverse of $\bar{L}$. ∎

Figure 8 compares the optimal value of $\lambda_{n}(L)$ to the one obtained by the uniform distribution, as the budget $c$ varies for a multiplex with two different random geometric network layers with 30 nodes. The optimal distribution gives smaller $\lambda_{n}$ for all budgets. We observe that after coalescing with $\lambda_{n-1}$, i.e. after the threshaold $c_{1}^{*}$, the multiplicity of $\lambda_{n}^{*}$ never decreases back to one. This phenomenon seems to persist in all other examples we have computed. Moreover, for $c>c_{1}^{*}$, it becomes impossible for a condition such as (17) to hold, since there will be generally no diagonal matrix $W$ satisfying multiple conditions corresponding to different eigenvectors of $L$. Therefore, $\lambda_{N}^{1}$ will be no longer attainable, and the optimal solution $\lambda^{*}_{n}$ enters a nonlinear regime for $c>c_{1}^{\star}$.

In Figure 9, we illustrate the behavior of the optimal weight distribution for this example, when varying the budget. For $c<c_{1}^{*}$, the optimal weight distribution is highly nonuniform in that the total budget is assigned to one node, i.e. to node numbered 11, while the others experiencing zero weight. Inspecting the eigenvector $v_{N}^{1}$ corresponding to the largest eigenvalue $\lambda_{N}^{1}$ of the individual layers reveals that $v_{N}^{1}(11)=0.0013$, and this is the smallest absolute value among the entries of $v_{N}^{1}$. Therefore we see that even though (17) is not exactly satisfied, i.e. the nodal set of $v_{N}^{1}$ is empty, $\lambda_{n}^{*}$ is still close to $\lambda_{N}^{1}$.

Figure 9 also indicates that the optimal weight distribution wants to become uniform with increasing $c$. This seems to be a general phenomenon and we provide an explanation next. First, the uniform weight distribution imposes an upper-bound on $\lambda_{n}^{*}$, i.e., $\lambda_{n}^{*}\leqslant\lambda_{max}(L_{ave})+2c/N$ for large $c$. Second, $\lambda=2c/N$ is always a lowerbound obtained from substituting $v_{1}=-v_{2}\equiv 1$ in (14). Therefore, for large values of $c$, $\lambda_{n}^{*}$ is restricted as follows

[TABLE]

Scaling the elements of the Laplacian matrix by $1/c$, scales the eigenvalues accordingly. Moreover, as $c$ goes to infinity, the scaled multiplex behaves more and more like a perfect matching bipartite graph, hence $\lambda_{n}$ tends to be twice the largest weight of the interlayer edges. Consequently, in the limit $\lambda_{n}^{*}$ approaches the uniform weight distribution and the lower bound in (30).

However, for large $c$ the largest eigenvalue $\lambda_{n}$ has multiplicity, while the uniform weight distribution leads to single largest eigenvalue for large $c$ (Solé-Ribalta et al., 2013). Therefore, there is a gap, albeit vanishing, between the optimal and the uniform weight distribution for large budgets shown in Figure 9(d-e) (for more results see SM .7). This becomes clearer in the next section when we examine the dual formulation.

III.1 Dual formulation and embedding of minimizing $\lambda_{n}$

The dual problem of (13) is:

[TABLE]

Propositions 2 and 3 also hold for (31), thus strong duality is satisfied by the primal and dual problems (13) and (31). By considering $Y$ as a Gram matrix $Y=V^{T}V$, where $V\in\mathbb{R}^{n\times n}$, then the problem (31) is equivalent to the following embedding problem:

[TABLE]

We interpret the column vectors $v_{i}^{*}$ of the matrix $V^{*}$ as coordinates for the position of each node in the corresponding embedding into $\mathbb{R}^{n}$.

The following result is similar to Proposition 4.

Proposition 9**.**

Projecting an optimal embedding $v_{1}^{*},...,v_{n}^{*}$, solving (32), onto a one-dimensional subspace, yields an eigenvector for the largest eigenvalue $\lambda_{n}^{*}$.

In the embedding problem (32), although not an explicit constraint, the barycenter is forced to stay at the origin. This follows from Proposition 9 and the fact that the eigenvector coresponding to $\lambda_{n}(L)$ is perpendicular to the constant ones vector $\bm{1}$.

Moreover, we have the following observation.

Remark 10**.**

Proposition 9 implies that the multiplicity of $\lambda_{n}(L)$ is an upper-bound on the dimension of the embedding problem (32).

Using Proposition 9 and Theorem 7, for $c\leqslant c_{1}^{*}$ the eigenvector corresponding to $\lambda_{n}^{*}$ is $v_{n}=\left({v_{N}^{1}}^{T},0\right)^{T}$, and therefore the following result holds.

Proposition 11**.**

For budget values up to the threshold $c^{\star}$, as defined in Theorem 7, the optimal solution of the embedding problem (32) is one-dimensional and, up to a rotation, is given by

[TABLE]

where $\gamma$ is a constant and $e_{1}\in\mathbb{R}^{n}$ is the first standard basis vector.

An illustration of (33) is given in Figure 10b.

Similarly, for very large values of $c$, we have the following statement.

Proposition 12**.**

For very large values of $c$, if $v_{1}^{*},...,v_{n}^{*}$ is the optimal solution of the embedding problem (32), then $v_{i}^{*}\approx-v_{N+i}^{*}$, for $i=1,...,N$.

Figure 10 shows the optimal embedding for a multiplex with two Watts-Strogatz layers. According to Remark 10, the embedding dimension remains bounded by the multiplicity of $\lambda_{n}^{*}$ in Figure 10a. The embedding for $c=1<c^{\star}$, in 10b shows a 1-dimensional embedding where the nodes of the layer with the largest $\lambda_{N}$, in this case $G_{1}$, are distributed along a line centered on the origin, while the layer $G_{2}$ is concentrated at the origin. Consequently, the optimal embeddings for small budgets $c<c^{\star}$ are based on (33) and are driven by $\lambda_{N}^{1}$ and its corresponding eigenvector $v_{N}^{1}$.

For budgets slightly or moderately above $c_{1}^{*}$, the embedding of the individual layers starts to expand in the plane, see Figures 10c and 10d. However, the optimal embedding is still influenced by $\lambda_{N}^{1}$ and $v_{N}^{1}$ so the nodes of $G_{2}$ with smaller largest eigenvalue unfold slowly around the origin, while the nodes of $G_{1}$ are still related to the components of $v_{N}^{1}$.

For larger budgets $c$, the optimal embedding expands into 3-D space. Figure 10e shows the expansion of the nodes of $G_{2}$ in 3-D space. In Figure 10f, we display a 2-D projection of the embedding and see that the nodes of the two layers are mixed and become more homogeneous, in the sense that each interlayer link in the embedding crosses the origin at the midpoint, thus confirming the claim in Proposition 12.

IV Minimizing spectral width $\lambda_{n}-\lambda_{2}$

The problem of distributing the total budget $c$ on the inter-layer links to minimize the spectral width of the Laplacian matrix is defined as

[TABLE]

where $L_{0}=\sum_{\{i,j\}\in E_{1}\cup E_{2}}L_{ij}$. In terms of Rayleigh quotients, (34) becomes

[TABLE]

Lemma 13**.**

Assume that $\lambda_{N}^{1}$ is the greatest largest Laplacian eigenvalue of the layers. Then, $H(c)$ is bounded below as follow

[TABLE]

and (36) is a strict inequality for $c\neq 0$.

Proof.

We proved $\lambda_{2}\leq 2c/N$ in (Shakeri et al., 2016) and showed in (18) that $\lambda_{n}\geq\lambda_{N}^{1}$. Thus the lowerbound (36) is trivial; furthermore, the equality $H(c)=\lambda_{N}^{1}-2c/N$ is possible only if $\lambda_{n}=\lambda_{N}^{1}$ and $\lambda_{2}=2c/N$. Therefore, the weight distribution must satisfy the weight distribution conditions corresponding to the problems of maximizing $\lambda_{2}$ and minimizing $\lambda_{n}$ below the thresholds $c^{*}$ and $c^{*}_{1}$ respectively. However, a uniform weight distribution can not satisfy (17) for $v_{N}^{1}\neq 0$ and $c\neq 0$.

∎

Lemma 13 implies that, the lower bound (36) is not reachable for $c>0$. However, since the lower bound is reachable for the extreme case of $c=0$, it follows from continuity, that the solution of (34) approaches the lower bound (36) for small budgets. That is

[TABLE]

Remark 14**.**

Since the algebraic connectivity $\lambda_{2}$ remains bounded for large budgets, according to (5), while the largest eigenvalue $\lambda_{n}$ grows unbounded after the threshold $c^{\star}$, it follows minimizing the gap $\lambda_{n}-\lambda_{2}$ is equivalent to minimizing $\lambda_{n}$ in such condition. Therefore, for large $c$, the solution to (34) approaches a uniform weight distribution. Moreover, according to (5) and (30), the minimum gap is bounded as follows

[TABLE]

In Figure 11, we show the spectral width in a multiplex of two random geometric networks with 30 nodes. We see that the minimized gap is smaller than that of the uniform weight distribution in all budget regimes. The optimal gap can be investigated in three regimes of small, moderate, and large $c$. For small $c$, $c<c^{*}$, the gap approaches the linear asymptote (37) so that it begins from $\lambda_{N}^{1}$ at $c=0$ and decreases with slope $2c/N$. Then, it remains (almost) constant for moderate $c$, $c^{*}<c<c^{\star}$, and it increases (approximately) with slope $2c/N$ for large $c$, $c>c^{\star}$.

In Figure 12, we compare different gaps associated with different optimization problems for a multiplex of two random geometric networks with 30 nodes. According to Remark 14, for large $c$, the gap obtained by minimizing $\lambda_{n}-\lambda_{2}$ approaches the gap obtained for the minimizer when minimizing only $\lambda_{n}$. For moderate $c$, this relation is inverted. Finally, for small $c$, the gap obtained for the weights that maximize the algebraic connectivity is the same as the one for uniform weights. while it grows with steep slope after the corresponding threshold $c^{*}$. The largest gap is thereby associated with the problem of maximizing the algebraic connectivity.

The above observations are supported by studying the optimal weight distributions obtained from different problems (Figure 13). For small budgets, the weights that are minimizing the gap are more similar to the uniform weights for maximizing the algebraic connectivity. However, minimizing the largest eigenvalue corresponds to completely nonuniform weights. For large budgets, the weight distribution for minimizing the gap is similar to weights for minimizing the largest eigenvalue (for more on this see SM .8).

IV.1 The embedding problem associated to minimizing the spectral width

The dual problem of (34) is

[TABLE]

It is an easy exercise to show that the primal problem (34) and dual problem (39) have feasible solutions, and thus common finite optimal value. Moreover, strong duality holds and the optimal value is attainable.

By Gram representations $X=U^{T}U$ and $Y=V^{T}V$, we obtain the embedding problem as the following non-convex problem:

[TABLE]

The following proposition guarantees this combined embedding is directly related to the single embeddings (7) and (32). Namely, the embedding problem (40) has the same projection properties as in Propositions 4 and 9.

Proposition 15**.**

The projections of the optimal $u$ and $v$ vectors onto one-dimensional subspaces yield eigenvectors to the second eigenvalue $\lambda_{2}(L)$ and the largest eigenvalue $\lambda_{n}(L)$ respectively 111Proof is similar to SM .4, and follows from complementary slackness. .

The main consequence of Proposition 15 is that the multiplicities of $\lambda_{2}$ and $\lambda_{n}$ give upper bounds on the dimensionality of the $u$ and $v$ embeddings of (40), respectively. However, the dimensions of the combined embeddings do not necessarily match the individual ones, see SM .8.

V Conclusions

In this paper, we investigate the allocation of a budget $c$ on the interlayer links to optimize certain functions defined on the eigenvalues of the Laplacian matrix for multiplex networks. The primal problem, in terms of Rayleigh quotients, and the dual problem, in terms of embeddings, are useful to glean the optimal structure of the multiplex. Maximizing the algebraic connectivity, $\lambda_{2}$, requires a uniform weight distribution up to a threshold budget. While, for larger budgets, the optimal weights are generally non-uniform. The largest eigenvalue $\lambda_{n}$ remains constant if we invest the budget on the edges that correspond to the nodal lines of the layer with larger Laplacian spectral radius. Moreover, as we increase the budget, the optimal weights tend to become highly homogeneous, almost uniform. Using these findings from the separate problems, maximizing $\lambda_{2}$ and minimizing $\lambda_{n}$, we analyze the interlayer weights that minimize the spectral width $\lambda_{n}-\lambda_{2}$. There we see that for small budgets the problem behaves similarly to maximizing the algebraic connectivity and, for large budgets, minimizing the spectral radius.

Acknowledgment

P. Poggi-Corradini is supported by NSF grant n. 1515810.

Supplemental materials

.1 Dual formulation of (8)

The problem in (8) is a standard convex semi-definite program (SDP) (Boyd and Vandenberghe, 2004, Chapter 4). The Lagrangian will be

[TABLE]

and the dual is

[TABLE]

The two SDP problems in (8) and (42), form a dual pair that satisfy the weak and strong duality, i.e., for any feasible $\hat{\textbf{w}}$ and $X$:

[TABLE]

.2 Proof of Proposition 2

Proof.

Let

[TABLE]

where

[TABLE]

with $\bm{h}\in{\mathbb{R}^{n}},\lVert\bm{h}\rVert=1$. If the constants $\alpha_{i}$ and $\beta_{i}$ exist such that

[TABLE]

then we can choose $\xi\leq\min\left\{-\left(\alpha_{i}-\beta_{i}\right)^{2}\left|ij\in E_{3},i\in V_{1},j\in V_{2}\right.\right\}$ and $X$ is feasible for the dual problem. Since $X$ is a Gram matrix, it is positive semidefinite. It is not a difficult task to show that Equation (46) is solvable. Indeed one solution is simply $\alpha_{i}=-\beta_{i}=\dfrac{1}{\sqrt{n}}$. ∎

.3 Proof of Proposition 3

Proof.

Since the primal problem (3) is a convex optimization (Shakeri et al., 2016), to show strong duality it is sufficient to show the Slater’s constraint qualification is satisfied. First, it is observed that $\left(\hat{\lambda}<0,\hat{\mu}\geq 0,\hat{\omega}>0\right)$ is a strictly feasible primal solution whenever $\sum_{ij\in E_{3}}\omega_{ij}=c$. The feasible primal solution $\left(\hat{\lambda}<0,\hat{\mu}\geq 0,\hat{\omega}>0\right)$ and the existence of dual solution by Proposition 2 gives rise to strong duality by the Slater’s qualification condition. Since none of feasible sets are empty, the optimal value is finite. Therefore the dual attains its optimal solution. Moreover, the primal constraint gives $\omega_{ij}\leq c$. Thus the weights remain in a compact subsets. ∎

.4 Proof of Proposition 4

Proof.

Let $U=\left[u_{1}\dots u_{n}\right]$ be an optimal embedding of (7) and $\omega_{ij}$ be the corresponding optimal weights leading to the algebraic connectivity $\lambda_{2}$ in (3). Then $X=U^{T}U$ is an optimal solution of (6). Since by KKT complementarity condition for the first inequality in (3) we have $\left<X,\sum_{ij\in E_{3}}w_{ij}L_{ij}+L_{0}+\mu\bm{e}\bm{e}^{T}-\lambda_{2}I\right>=0$, and by the second equality in (6) $\left<X,\bm{e}\bm{e}^{T}\right>=0$, it follows

[TABLE]

where we use the condition $\text{trace}\left(U^{T}UA\right)=\text{trace}\left(UAU^{T}\right)$. The above condition shows that $\text{trace}\left[U\left(\sum_{\{i,j\}\in E_{3}}\omega_{ij}L_{ij}+L_{0}-\lambda_{2}I\right)U^{T}\right]=0$, so that

$\sum_{i=1}^{n}y_{i}^{T}\left(\sum_{\{i,j\}\in E_{3}}\omega_{ij}L_{ij}+L_{0}-\lambda_{2}I\right)y_{i}=0$ with $y_{i}$ the i-th row of $U$. Since $\lambda_{2}$ is the smallest nonzero eigenvalue, we know that each element $y_{i}^{T}\left(\sum_{\{i,j\}\in E_{3}}\omega_{ij}L_{ij}+L_{0}-\lambda_{2}I\right)y_{i}\geq 0$. Therefore, $y_{i}^{T}\left(\sum_{\{i,j\}\in E_{3}}\omega_{ij}L_{ij}+L_{0}-\lambda_{2}I\right)y_{i}=0$. This indicates that each row of $U$, or column of $U^{T}$, and accordingly the vector $\nu=U^{T}p$ with $p$ arbitrary vector, is in the eigenspace of $L=\sum_{\{i,j\}\in E_{3}}\omega_{ij}L_{ij}+L_{0}$ to $\lambda_{2}$. ∎

.5 Proof of Lemma 5

For $c<c^{*}$, the second eigenvalue of the supra-Laplacian is simple, and the corresponding eigenvector is a one-dimensional subspace given by the Fiedler vector $\bm{v}=\frac{1}{\sqrt{n}}\left[\bm{e};-\bm{e}\right]$ (Shakeri et al., 2016). Then, since by Proposition 4 the projection of optimal embedding onto any arbitrary one-dimensional subspace is parallel to $\bm{v}$, i.e. $\exists\hskip 2.84526pta>0\hskip 2.84526pt\text{s.t.}\hskip 2.84526pt\left[\hat{u_{1}},\dots,\hat{u_{n}}\right]^{T}\bm{p}=a\bm{v}\hskip 8.53581pt\forall\hskip 2.84526pt\bm{p}\in\mathbb{R}^{n}$, it can result that the optimal embedding takes the form (10).

.6 Separator-shadow theorem

Theorem 16**.**

Let the weights $w_{ij}\geq 0$ and points $\hat{u}_{i}\in\mathbb{R}^{n},\ i\in V,$ be optimal solutions of (8) and (9), respectively. Let $S$ be a separator of $G_{w}$ partitioning the graph as $V=S\cup C_{1}\cup C_{2}$ with no edge between $C_{1}$ and $C_{2}$. Then, for at least one $j\in\{1,2\}$

[TABLE]

That means, there is at least one $j\in\{1,2\}$ such that, the straight line between the origin and each $\hat{u}_{i},i\in C_{j},$ intersects the convex hull of the points in $S$.

First, the following Lemma 17 due to Fiedler (see Theorem 3.3 in (Fiedler, 1975)) plays a crucial role in characterizing the graph structure based on a Fiedler vector, i.e. an eigenvector to the second smallest eigenvalue. First, denoting a Fiedler vector by $y=y(i)=[y_{i}],\ \forall i\in V$, the coordinates of $y$ can be assigned to the vertices of $G$. This is called a characteristic valuation.

Lemma 17**.**

Let $G=\left(V,E\right)$ be a finite connected graph with positive weights $w_{ij}$. Let $y_{i}$ be a characteristic valuation of $G$, and for any $\alpha\geq 0$, define $V_{\alpha}=\{i\in V|y_{i}+\alpha\geq 0\}$. Then the subgraph $G(\alpha)$ induced by $G$ on $V_{\alpha}$ is connected.

Now, having already shown Proposition 4, we can conclude the following Lemma 18 for multiplex networks.

Lemma 18**.**

Let the weights $w_{ij}\geq 0$ and points $\hat{u}_{i}\in\mathbb{R}^{n},\ i\in V,$ be optimal solutions of (8) and (9), respectively. Let $S$ be a separator of $G_{w}$ partitioning the graph as $V=S\cup C_{1}\cup C_{2}$ with no edge between $C_{1}$ and $C_{2}$. Suppose there is a normalized $b\in\mathbb{R}^{n}$ and $\beta>0$ defining a hyperplane $b^{T}x=\beta$ within the subspace $\text{span}\{\hat{u}_{i},i\in V\}$ that separates all points of $S$ from at least one point $\hat{i}\in C_{1}$, namely

[TABLE]

Then

[TABLE]

Proof.

By Proposition 4 we know that the vector $-\left[b^{T}\hat{u}_{1},\dots,b^{T}\hat{u}_{n}\right]^{T}$ is an eigenvector to $\lambda_{2}\left(L\left(G_{w}\right)\right)$. Then, setting $\alpha=\beta$ in Lemma 17, it follows that the subgraph $G\left(\beta\right)=\left(V_{\beta},E_{w}\right)$ with $V_{\beta}=\{i\in V:b^{T}\hat{u}_{i}\leq\beta\}$ is connected. Therefore, since there is no edge between $C_{1}$ and $C_{2}$, the set $V_{\beta}$ can not simultaneously contain nodes from both $C_{1}$ and $C_{2}$, and since it already contains a node form $C_{1}$, i.e. $\hat{i}\in V_{\beta}$, it follows there is no point of $C_{2}$ in $V_{\beta}$. ∎

Lemma 18 means that, of $C_{1}$ and $C_{2}$, only at most one part can be separated by the hyperplane $b^{T}x=\beta$ from $S$ (see Figure 14). This is the key to reach separator-shadow theorem.

Proof of Theorem 16.

. It is supposed that the origin is not contained in the convex hull of the points in $S$, and none of $S$, $C_{1}$, and $C_{2}$ are empty; otherwise the theorem holds trivially.

The proof is by a contradiction argument. If the theorem does not hold, then there are points $\hat{u}_{i}$ and $\hat{u}_{j}$ with $i\in C_{1}$ and $j\in C_{2}$ such that $\text{conv}\left[0,\hat{u}_{i}\right]\cap\mathcal{S}=\text{conv}\left[0,\hat{u}_{j}\right]\cap\mathcal{S}=\emptyset$, with $\mathcal{S}=\text{conv}\{\hat{u}_{s}:s\in S\}$ (Figure 15). Then, convexity shows for any $k\in\{i,j\}$ there is a hyperplane in the space spanned by the solution of (9) that separates the line segment $\text{conv}\left[0,\hat{u}_{k}\right]$ from $\mathcal{S}$, i.e. there are $b_{k}\in\text{span}\{\hat{u}_{r}:r\in V\}$ and $\beta_{k}>0$, such that $b_{k}^{T}x\geq\beta_{k}\ \forall x\in\mathcal{S}$ and $b_{k}^{T}x<\beta_{k}\ \forall x\in\text{conv}\left[0,\hat{u}_{k}\right]$.

Next it is seen that there can be found a convex combination of the two hyperplanes by an $\alpha\in\left[0,1\right]$ such that, for $b\left(\alpha\right)=\left(1-\alpha\right)b_{i}+\alpha b_{j}$ and $\beta\left(\alpha\right)=\left(1-\alpha\right)\beta_{i}+\alpha\beta_{j}$, the half-space $\mathcal{H}=\{x:b\left(\alpha\right)^{T}x<\beta\left(\alpha\right)\}$ contains points of both $C_{1}$ and $C_{2}$. This is because $\mathcal{H}$, that includes origin for all $\alpha\in\left[0,1\right]$, contains $\hat{u}_{i}\in C_{1}$ for $\alpha=0$ and $\hat{u}_{j}\in C_{2}$ for $\alpha=1$. Now, if for contradiction, when continuously varying $\alpha$ from [math] to $1$, there is no $\alpha$ for which $\mathcal{H}$ contains points from both $C_{1}$ and $C_{2}$, there should exist an $\alpha=\bar{\alpha}$ where $\mathcal{H}$ leaves $C_{1}$ before it meets $C_{2}$. In such condition, the hyperplane $b\left(\bar{\alpha}\right)^{T}x=\beta\left(\bar{\alpha}\right)$ would separate the origin from $\text{conv}\{\hat{u}_{r}:r\in V\}$, thus violating the equality constraint in (9).

The fact that $\mathcal{H}$ contains points from both $C_{1}$ and $C_{2}$ contradicts Lemma 18, and thus the points $\hat{u}_{i}$ and $\hat{u}_{j}$ assumed initially can not exist. ∎

The situation can be checked for the separator involving the node set $\{16,11,12,20,8,38\}$ in the multiplex network of Figure 16 .

.7 More results on maximizing the spectral radious of the Laplacian matrix

In Figures 17a and 17b, we plot, respectively, the largest eigenvalue and the second eigenvalue associated with different optimization problems. In Figure 17a, while maximizing the algebraic connectivity $\lambda_{2}$ generates the largest eigenvalue particularly after the threshold $c^{*}$, the problems of minimizing $\lambda_{n}$ and minimizing $\lambda_{n}-\lambda_{2}$ lead to similar results for the largest eigenvalue with some difference in small $c$. On the other hand, Figure 17b illustrates that the algebraic connectivity resulting from minimizing the gap is approximately equal to that of maximizing $\lambda_{2}$ for small budgets and that of minimizing $\lambda_{n}$ for large budgets.

For moderate budgets, the algebraic connectivity of minimizing the gap is smaller than the value obtained by uniform weight distribution. Moreover, the problem of minimizing $\lambda_{n}$ generates the minimum algebraic connectivity at all. A further result is that, from perspective of the algebraic connectivity in Figure 17b, the problem of minimizing the gap works based on (is approximately equivalent to) maximizing the algebraic connectivity for small budgets smaller than the threshold $c^{*}$, while it operates through (is approximately equivalent to) minimizing the largest eigenvalue $\lambda_{n}$ for large $c$, particularly after the threshold $c^{\star}$. For the largest eigenvalue in Figure 17a, on the other hand, minimizing the gap operates similar to minimizing the largest eigenvalues for all budgets. The full spectra of supra-Laplacians associated with different primal problems are seen in Figure 18.

.8 Dimensionality of the combined embedding associated to minimizing the spectral width

Figure 19 illustrates the dimensions of $u$- and $v$-embeddings in different regimes of the budget for a multiplex of two Watts-Strogatz networks, each with 30 nodes.

Moreover, we show the corresponding embeddings for $\lambda_{2}$ and $\lambda_{n}$ in minimum width problem in Figure 20(a-b) for a small budget $c=5$ together with their counterparts in maximum $\lambda_{2}$ and minimum $\lambda_{n}$ problems in Figure 20(c-d). Whilst $u$- and $v$-embeddings are one dimensional they are perturbation of the corresponding individual problems, with more perturbation in $u$-embedding, and more similarity in $v$-embedding. This can be explained by the less rigid conditions in (17) compared to the optimal condition for maximizing algebraic connectivity.

Figure 21a and 20c illustrates the embeddings for $c=20$. The $u$-embedding and the corresponding embedding of maximizing $\lambda_{2}$ are both two dimensional. However, Figure 21 shows a 1-D embedding for $v$-embedding versus two dimensional embedding for minimizing $\lambda_{n}$.

Figure 22 shows the embeddings for $c=50$ with one and two dimensional $u$- and $v$-embeddings are comparable with three dimensional embeddings of individual problems.

.9 Individual network models

In the numerical simulations discussed throughout the paper, we use four different graph models for individual networks:

Barabási-Albert scale-free network (BA). New nodes are attached to a specified number of already existing nodes in a preferential attachment fashion. For nodes number large enough, this method ensures the emergence of power-law behavior observed in many real-world networks (Barabási and Albert, 1999).

Erdős-Rényi (ER). Starting with a complete graph for the given number of nodes, the edges are randomly deleted according to a specified probability (Bollobás, 1998).

Geometric network (Geo). A set of nodes, picked randomly in a specified interval, are connected by an edge if the Euclidean distance is up to a definite value (Penrose et al., 2003).

Watts-Strogatz (WS). First, all nodes are connected to their immediate neighbors according to a fixed degree specified. Then, all existing links are rewired with a given probability, which produces graphs with low average hop count yet high clustering coefficient, which mimics the small-world property found in real-world networks (Watts and Strogatz, 1998).

.10 Correlation with centrality measures

We have observed that maximum algebraic connectivity is achieved with uniform wights when $c\leq c^{*}$. For larger budgets the weight distribution is generally not uniform. To illustrate that optimal weights are not so simple as to be exclusively determined based on common centrality measures, we examine the weight distribution for maximizing algebraic connectivity corresponding to Figure 7. We examined several common centrality measures such as Degree, Eigenvector, and Page Rank, as well as Fiedler vector components, to evaluate nodes importance. Here, only results of degree centrality and Fiedler vector are shown in Figure 23, since no additional results were observed by other measures. Figures 23a and 23b report no correlation of optimal weights with degree centrality of individual network components. There is also no correlation in Figure 23d between optimal weights and Fiedler vector components of the graph with larger connectivity; here $G_{2}$. However, some positive correlation is observed between optimal weights and Fiedler vector components of the graph with smaller connectivity; here $G_{1}$, i.e. the subgraph first unfolds in Figure 7. Therefore, for budget values beyond the threshold $c^{*}$, optimal weights prove positive correlation with Fiedler vector components of the subgraph with smaller algebraic connectivity so that, generally, the higher the absolute value of Fiedler vector component corresponding to a node in this subgraph, the larger the weight assigned to it; or, the smallest weights are assigned to those nodes with smallest Fiedler vector component and the largest weights are assigned to those nodes with largest Fiedler vector component . The layer with larger connectivity hardly plays role in this regard. We remind that this condition is manifested for budget values just above $c^{*}$ and when the two subgraphs hold very different algebraic connectivity values. Thus, with distancing from these two conditions correlation between optimal weights and Fiedler components of the subgraph with smaller algebraic connectivity dies away. Finally, we show in Figure 24 the results corresponding to the larger network investigated in Figure LABEL:fig:EmbLrg. Again, only some positive correlation can be observed with Fiedler vector of subgraph $G_{1}$ with smaller algebraic connectivity.