A class of randomized Subset Selection Methods for large complex networks

TL;DR

This paper introduces randomized subset selection methods to efficiently analyze large complex networks by approximating their spectra and structure, significantly reducing computational time while maintaining accuracy.

Contribution

It proposes novel randomized heuristics for subset selection in large networks, improving speed and scalability over deterministic methods.

Findings

Randomized methods are faster than deterministic counterparts.

QR-based randomized subset selection is the quickest approach.

Methods effectively analyze large, sparse, and dynamic networks.

Abstract

Most of the real world complex networks such as the Internet, World Wide Web and collaboration networks are huge; and to infer their structure and dynamics one requires handling large connectivity (adjacency) matrices. Also, to find out the spectra of these networks, one needs to perform the EigenValue Decomposition(or Singular Value Decomposition for bipartite networks) of these large adjacency matrices or their Laplacian matrices. In the present work, we proposed randomized versions of the existing heuristics to infer the norm and the spectrum of the adjacency matrices. In an earlier work [1], we used Subset Selection (SS) procedure to obtain the critical network structure which is smaller in size and retains the properties of original networks in terms of its Principal Singular Vector and eigenvalue spectra. We now present a few randomized versions of SS (RSS) and their time and space complexity calculation on various benchmark and real-world networks. We find that the RSS based on using QR decomposition instead of SVD in deterministic SS is the fastest. We evaluate the correctness and the performance speed after running these randomized SS heuristics on test networks and comparing the results with deterministic counterpart reported earlier. We find the proposed methods can be used effectively in large and sparse networks; they can be extended to analyse important network structure in dynamically evolving networks owing to their reduced time complexity.

Figures

Tables

Table 1. Table 1: This table shows the theoretical time complexity involved in all the algorithms proposed in the paper.

Algorithm	Matrix multiplication	SVD/QR
Algorithm 1	-	$𝒪\left(m{n}^2\right)$
Algorithm 2	$𝒪\left(mnq\right)$	$𝒪\left(n{q}^2\right)$
Algorithm 3	$𝒪\left(mnq\right)$	$𝒪\left(2n{q}^2-\frac{2}{3}{q}^3\right)+𝒪\left(n{q}^2\right)$
Algorithm 4	$𝒪\left(mnq\right)$	$𝒪\left(2n{q}^2-\frac{2}{3}{q}^3\right)$
Algorithm 5	$𝒪\left(mnq\right)+𝒪\left(n{q}^2\right)$	$𝒪\left(q^3\right)$

Table 2. Table 2: The table shows the numerical time complexity involved in subset selection procedure for all the algorithms proposed in the paper for six networks.

Algorithm	BA	Drosophila	ER	Friendship	Power Grid	US Air
Algorithm 1	1.014	4.895	1.148	5.165	87.408	0.060
Algorithm 2	0.371	1.761	0.380	1.978	32.546	0.034
Algorithm 3	0.392	2.097	0.434	2.306	37.424	0.036
Algorithm 4	1.109	4.116	1.185	4.364	56.132	0.057
Algorithm 5	0.281	1.268	0.252	1.441	55.639	0.044

Table 3. Table 3: A table of SS results on model networks and weighted and unweighted real networks examples. The real networks were downloaded from KONECT [ 21 ] and model network types were generated using python module Networkx [ 22 ] . (V, E) represents the vertices and the edges in the networks. The columns loss1, loss2, loss3, loss4 and loss5 represent the Frobenius norm differences of main networks and the corresponding subsets obtained through Deterministic SS, RSS1, RSS2, RSS3 aand RSS4 respectively. CS represents cosine similarity between PSVs of main network adjacency and all the subsets. It is found to be greater than the value in the column for all the subsets.

Type	Networks	(V, E) of network	q	(V, E) of SS network	loss1	loss2	loss3	loss4	loss5	$CS$
Weighed, real	US Air	(332, 2126)	166	(890,1635)	0.011	0.010	0.010	0.010	0.010	$>$0.99
	Les Miserables	(77, 254)	38	(38, 141)	0.017	0.028	0.028	0.028	0.025	$>$0.99
	Train Bombing	(64, 243)	32	(32, 123)	0.100	0.100	0.100	0.100	0.105	$>$0.99
Unweighted, real	Karate	(34, 78)	20	(20, 47)	0.156	0.164	0.164	0.164	0.152	$>$0.94
	Cat Brain	(65, 730)	32	(32, 247)	0.236	0.232	0.232	0.232	0.219	$>$0.99
	Drosophila	(1781, 9016)	890	(890, 7026)	0.057	0.058	0.058	0.058	0.056	$>$0.99
	Power Grid	(4941, 6594)	2470	(2470, 2863)	0.166	0.161	0.161	0.161	0.160	$>$0.985
	Jazz Musicians	(198, 2742)	96	(96, 947)	0.232	0.214	0.214	0.214	0.220	$>$0.98
	Friendship	(1858, 12534)	929	(929, 7618)	0.114	0.113	0.113	0.113	0.109	$>$0.99
Unweighted, model	Barabasi Albert	(1000, 2991)	500	(500, 1418)	0.150	0.151	0.151	0.151	0.151	$>$0.99
	Erdos Renyi	(1000, 7558)	500	(500, 2630)	0.230	0.234	0.234	0.234	0.233	$>$0.98
	Power Law	(1000, 1360)	500	(500, 947)	0.082	0.087	0.087	0.087	0.082	$>$0.88

Equations

$$\parallel A_{m\times n}{\parallel }_F=\sqrt{\sum _{i=1}^{i=m}\sum _{j=1}^{j=n}|a_{ij}{|}^2}=\sqrt{\sum _{i=1}^{min(m,n)}{\sigma }_i^2}$$

$$A=Q\left(\genfrac{}{}{0ex}{}{R}{0}\right)P^T,m\ge n$$

$$|r_{11}|\ge |r_{22}|\ge |r_{33}|\ge .....\ge |r_{nn}|$$

$$|r_{ll}|\geq\|R_{l:j,j}\|_{2}\quad\mbox{for $j=l+1,\ldots,n$.}$$

$$(1-ϵ)\parallel x-y{\parallel }^2\le \parallel f(x)-f(y){\parallel }^2\le (1+ϵ)\parallel x-y{\parallel }^2$$

$$\mathcal{A}P=[{\mathcal{A}}_1,{\mathcal{A}}_2]$$

$$\mathcal{A}=\mathcal{U}\Sigma {\mathcal{V}}^T.$$

$$\sum _{i=1}^q{\sigma }_i^2\simeq \sum _{i=1}^r{\sigma }_i^2=\Vert \mathcal{A}{\Vert }_F^2$$

$${\bar{\mathcal{A}}}_{q\times n}={\Omega }_{q\times m}{\mathcal{A}}_{m\times n}$$

$${\bar{\mathcal{A}}}_{q\times n}={\Omega }_{q\times m}{\mathcal{A}}_{m\times n},k\ll m$$

$${\mathcal{U}}_{\bar{\mathcal{A}}}{\Sigma }_{\bar{\mathcal{A}}}{\mathcal{V}}_{\bar{\mathcal{A}}}^T\approx \bar{\mathcal{A}}$$

$${\mathcal{V}}_{\bar{\mathcal{A}}}=\text{eigenvector}({\bar{\mathcal{A}}}^T\bar{\mathcal{A}})$$

$$\bar{\mathcal{A}}^{T}\,\bar{\mathcal{A}}=\big{(}\mathcal{A}^{T}\Omega^{T}\big{)}\,\big{(}\Omega\mathcal{A}\big{)}\approx\mathcal{A}^{T}\mathcal{A}$$

$$\mathcal{V}_{\bar{\mathcal{A}}}=\text{eigenvector}\big{(}\bar{\mathcal{A}}^{T}\,\bar{\mathcal{A}}\big{)}\approx\text{eigenvector}\big{(}\mathcal{A}^{T}\,\mathcal{A}\big{)}|_{\text{top}\ q\,\text{basis}}$$

$${Q_{\bar{\mathcal{A}}}}_{q\times q}{R_{\bar{\mathcal{A}}}}_{q\times n}\approx \bar{\mathcal{A}}$$

$${{\mathcal{U}}_{R_{\bar{\mathcal{A}}}}}_{q\times q}{{\Sigma }_{R_{\bar{\mathcal{A}}}}}_{q\times q}{{\mathcal{V}}_{R_{\bar{\mathcal{A}}}}^T}_{q\times n}\approx R_{\bar{\mathcal{A}}}$$

$$\bar{\mathcal{A}}\approx\big{(}Q\,\mathcal{U}_{R_{\bar{\mathcal{A}}}}\big{)}\,\Sigma_{R_{\bar{\mathcal{A}}}}\,\mathcal{V}^{T}_{R_{\bar{\mathcal{A}}}}$$

$${\bar{\mathcal{A}}}^T\approx {\mathcal{V}}_{\bar{\mathcal{A}}}{\Sigma }_{\bar{\mathcal{A}}}^T{\mathcal{U}}_{\bar{\mathcal{A}}}^T$$

$$Q_{{\bar{\mathcal{A}}}^T}{R}_{{\bar{\mathcal{A}}}^T}\approx {\bar{\mathcal{A}}}^T$$

$${\mathcal{V}}_{\bar{\mathcal{A}}}\approx Q_{{\bar{\mathcal{A}}}^T}$$

$$Y_{q\times q}=\bar{\mathcal{A}}{\bar{\mathcal{A}}}^T={\mathcal{U}}_{\bar{\mathcal{A}}}{\Sigma }_{\bar{\mathcal{A}}}^2{\mathcal{U}}_{\bar{\mathcal{A}}}^T$$

$$Y\,\mathcal{U}_{\bar{\mathcal{A}}}=\Big{(}\bar{\mathcal{A}}\,\bar{\mathcal{A}}^{T}\Big{)}\mathcal{U}_{\bar{\mathcal{A}}}=\mathcal{U}_{\bar{\mathcal{A}}}\,\Sigma^{2}_{\bar{\mathcal{A}}}$$

$$YX=\Lambda X$$

$${\mathcal{U}}_{\bar{\mathcal{A}}}\approx X$$

$${\Sigma }_{\bar{\mathcal{A}}}\approx \sqrt{\Lambda }$$

$${\mathcal{V}}_{\bar{\mathcal{A}}}={\bar{\mathcal{A}}}^T{\mathcal{U}}_{\bar{\mathcal{A}}}{\Sigma }_{\bar{\mathcal{A}}}^{-1}\approx {\bar{\mathcal{A}}}^{T}X{\Lambda }^{-1}$$

$$CS=cos(\theta )=\frac{{\mathcal{A}}^1.{\mathcal{A}}_1^1}{\parallel {\mathcal{A}}^1\parallel \parallel {\mathcal{A}}_1^1\parallel }$$

$$[{\mathcal{B}}^T,{\mathcal{B}}_r^T]\equiv P{\mathcal{A}}_1$$

Full text

††preprint: APS/123-QED

I Introduction

Most of the real world complex systems can be modelled and studied as complex networks. Also, the spectra of these networks have a direct correlation with topological properties of the networks and hence the processes on them. For example, the Internet network [2] has power law distribution of eigenvalues with tail at larger eigenvalues. Its robustness to the random failure of nodes and absence of epidemic threshold is attributed to its scale-free topology. As a consequence, the eigenvalue spectra can be used as the fingerprint of the complex networks. In many spectacular works in complex networks theory, the spectra of complex networks [3], [4] adjacency matrix (A) and the Laplacian (L), has been successfully employed to infer synchronizability of complex networks [5], [6], the partition of network into modules or clusters [7], and controllability of epidemic spreading on the networks [8]. Similarly, the eigenvectors have been used to identify the most influential network nodes [9], [10] and spectral partitioning of networks into communities [11], [12], [13].

Any algorithm that uses network topology runs on polynomial time proportional to network size. To counter this, numerous efforts have been made to represent large complex networks in a reduced form such that their spectra are preserved. This reducibility is possible because of the presence of unimportant network structure, such that the network function is robust to shutdown or failure of its nodes and links. With a similar aim, we proposed using the Subset Selection (SS) algorithm [14] on the complex networks adjacency network [1] such that the obtained subset can be used to infer the spectra of original networks. This method is especially important for the cases of large complex networks where space and time complexity of analysing full adjacency matrices are too expensive. Also, although we have used the SS method to infer reduced representation of complex networks, it has been successfully used in a plethora of applications ranging from solving rank-deficient least square problems [14], in genetics [15], in wireless communication [16] and other information retrieval problems. Here we propose the randomized versions of the deterministic SS procedure and use them to infer the spectra of complex networks.

The Randomized SS procedures that we propose have reduced time and space complexity than the deterministic counterpart. The deterministic SS, uses first $q$ singular vectors (obtained by performing Singular Value Decomposition) of the matrix $\mathcal{A}$ (adjacency matrix for complex networks) and performs QR factorization with column pivoting on the matrix formed by these vectors to obtain a permutation vector ($P$) which is then used to reorder the columns of $\mathcal{A}$ in decreasing order of preference. The first $q$ columns of the reordered $\mathcal{A}$ form the subset. The randomized versions of SS firstly project the original matrix vectors on a lower dimensional space using a random matrix [17]. Following this, the SS procedure is performed on the projected vectors matrix. The randomized version runs on approximately half the time required by deterministic SS procedure. We also present a similar algorithm based on randomisation that offers reduced space complexity. The detailed calculation of time and space complexities for all methods are presented in the paper.

We show the results by comparing the singular value spectra obtained in the deterministic and randomized SS procedures among themselves and also with the spectra of the original network, for a predefined $q$. We also present plots of Principal Singular Vector (PSV) components for the subset obtained with deterministic and randomized procedures and capture their cosine similarities with the PSV of the original adjacency matrix. The cosine similarities that we get were fairly good depicting that the PSVs of subsets derived from the randomized versions of SS have remarkable overlap with the PSV of $\mathcal{A}$. This observation justifies that randomized subsets can competently represent the original network as far as its spectra and PSV is concerned. Also, the randomized versions select the same nodes as the important or influential ones, as chosen by the deterministic SS procedure. Hence the randomized SS offers the same benefits as deterministic SS on the complex networks in lesser time. Without loss of generality, the randomized SS algorithms can be applied to any dataset in matrix form apart from complex network adjacency matrix.

II Background

II.1 Mathematical preliminaries

• •

Rank : For a matrix with $m$ rows and $n$ columns, rank is defined as $r=min(m,n)$. If there is collinearity in the dataset (matrix), rank is the number linearly independent rows vectors or the number linearly independent columns vectors in the matrix, whichever is smaller.

• •

Energy of Matrix: Also given by the Frobenius norm, the energy of a matrix is the square root of the sum of the absolute square of all its elements. It is also given by square root of sum of squares of absolute singular values $\sigma_{i}$’s of the matrix.

[TABLE]

• •

Singular Value Decomposition (SVD): The Singular Value Decomposition is a matrix factorization technique and is a generalization of eigen decompostion to non-square matrices. For a matrix $A_{m\times n}$, the SVD is given by $M=U\Sigma V^{T}$, where $U_{m\times m}$ and $V_{n\times n}$ are orthogonal matrices with the left and the right singular vectors of $A$ and $\Sigma_{m\times n}$ is diagonal matrix with singular values $\sigma_{i}$ along its diagonal.

• •

QR: The QR decomposition of matrix $A$ is a matrix factorisation technique which decomposes matrix as $A=QR$, where $Q$ is an orthogonal matrix, and $R$ is an upper triangular matrix.

• •

QR-cp: QR factorization of $A_{m\times n}$with column pivoting is given by,

[TABLE]

where $Q$ and $R$ as before and $P$ is permutation matrix such that,

[TABLE]

and, for each $l$

[TABLE]

• •

Random Projection: It is a method used to reduce the dimensionality of data points lying in a Euclidean space. RP is generally used in handling and manipulating large datasets or manifolds to infer their intrinsic dimension or to know the principal directions.

• •

Johnson–Lindenstrauss lemma: The JL lemma states that the dataset can be projected to a much lower dimension than the original such that the distances between the points in the datasets in the original and the projected space is preserved. It is used extensively in the problems of compressed sensing, dimensionality reduction and graph embedding.

For any $0<\epsilon<1$ , a set $A$ of $m$ points in $N$ dimensional space, and a number $n>8ln(m)/\epsilon^{2}$ there exists a linear map $\textbf{f}:\mathbb{R}^{N}\rightarrow\mathbb{R}^{n}$,

[TABLE]

for all $x,y\in A$.

• •

Subset: A subset is the part of the original matrix with only significant columns such that it has the same Frobenius norm as that of the original matrix. The subset of a matrix exists if the singular value spectrum of the dataset is such that most of the matrix norm is made up of only a few singular values. The number of these dominant singular values is governed by the matrix rank $r$.

• •

Centrality: In complex network theory, the centrality of a node or an edge is an indicator of its importance in the network. Depending on the ways one defines importance there are many centrality measures regularly used such as degree centrality, betweenness centrality, eigenvector centrality, closeness centrality, PageRank centrality, etc.

II.2 Subset selection procedure

The subset selection [18] procedure is a well-known method to identify the essential column vectors of a data matrix $\mathcal{A}$. In our previous work, we showed the application of the subset selection procedure in large complex networks [1]. The basic idea of the deterministic subset selection (SS) method is to discard the redundant columns of the data matrix and keep those columns which have a maximum contribution in the matrix in terms of energy preservation. Mathematically the data matrix, $\mathcal{A}$ can be thought of as a collection of two blocks $[\mathcal{A}_{1},\mathcal{A}_{2}]$. Where $\mathcal{A}_{1}$ contains $q$ linearly independent columns which can approximately span the entire column space of $\mathcal{A}$ and $\mathcal{A}_{2}$, the collection of the redundant columns can be represented by the linear combination of the column vectors of $\mathcal{A}_{1}$.

It is understandable that the value of $q$ is dependent on the number of redundant columns. If the number of redundant columns is high, then $q$ will be decidedly less and vice-versa. Therefore the value of $q$ can be directly related to the numerical rank of the data matrix $\mathcal{A}$. Hence for a large data matrix, the size of the obtained subset $\mathcal{A}_{1}$ depends on the rank-deficiency of the data-matrix as it directly translates the value of $q$. To find the essential (non-redundant) block $\mathcal{A}_{1}$ and redundant block $\mathcal{A}_{2}$ respectively of $\mathcal{A}$, one has to compute the permutation matrix $P$ which helps to identify and separate out the two blocks as follows.

[TABLE]

Therefore the whole SS procedure [18] boils down to obtaining the optimal permutation matrix $P$. One can end up with different realisations of the $P$ matrices, but the optimal one will be decided based on the following criteria. The optimal $P$ should follow the following constraints.

The number of linearly independent columns ($q$)of $\mathcal{A}_{1}$ should represent the numerical rank of the data matrix $\mathcal{A}$. 2. 2.

The residual difference between the norm of the linear combination of $\mathcal{A}_{1}$ with $\mathcal{A}_{2}$ should be minimal.

These constraints mentioned above can be full-filled by studying the singular value decomposition(SVD) of $\mathcal{A}$. SVD [14] is one of the best numerical methods to obtain the critical basis of any arbitrary data-set. The singular value spectra provide the corresponding weights of the basis vectors; therefore it can be used to find out the value of $q$ as the numerical rank (normally decided based on the top-$q$ non-zero values).

Therefore the first step of the deterministic SS procedure is to obtain the factors $\mathcal{U}$, $\Sigma$, and $\mathcal{V}$ using SVD.

[TABLE]

The matrix $\mathcal{U}_{m\times r}$ represents eigenvectors of the left subspace of $\mathcal{A}$; the matrix $\Sigma_{r\times r}$ is a diagonal matrix with $r$ ($r=rank(\mathcal{A})=min(m,n)$) positive non-zero entries (known as singular values) arranged in descending order of magnitude i.e ($\sigma_{1}>\sigma_{2}>\sigma_{3}>...>\sigma_{r}$) and the matrix $\mathcal{V}_{r\times n}$ represents eigenvectors of the right subspace of $\mathcal{A}$.

To find out the value of $q$ we need to maximized the preservation of the Forbenius norm $\mathcal{A}$ such way that

[TABLE]

where $\|\mathcal{A}\|_{F}^{2}$ is the Frobenius norm of $\mathcal{A}$.

From eq.5 $\&$ 6, it is clear that $P$ can be obtained by permuting the first $q$ columns of the $\mathcal{V}$. This implies that finding out of $q$ important columns of $\mathcal{A}$ can be equivalently translated as to a problem of the finding out the optimal permutation of the corresponding columns in $\mathcal{V}^{T}$. Hence the second essential step of the deterministic SS procedure is to obtain $P$ based on truncated $\mathcal{V}^{T}$. The complete version of the deterministic subset selection procedure is described in Algo-1.

It is notable that one can apply the deterministic SS procedure for a system represented by a complex network. For that purpose one has to compute the SVD of the network’s adjacency matrix ($\mathcal{A}_{m\times m}$) and need to follow the steps described in Algo-1 to obtain ${\mathcal{A}_{1}}_{m\times q}$, where $\mathcal{A}_{1}$ is a $m\times q$ matrix representing important columns of original network adjacency matrix, $\mathcal{A}$.

II.3 Bottleneck of deterministic SS procedure

It is clear from the Algo-1, two essential numerical steps are involved in the deterministic SS procedure. First is the computation of SVD of the data matrix to obtain the set of basis vectors and the second one is to apply column-pivotal QR decomposition on the truncated right singular basis to get the optimal permutation matrix. For a large data matrix, the computation of the SVD is a practical issue. Generally, for a data matrix $\mathcal{A}_{m\times n}:m>n$, the theoretical time complexity of SVD is $\mathcal{O}(mn^{2})$. Time complexity will also increase exponentially with an increase in the size of the data matrix. Also, the first step in Algo-1 involves computation of the full set of right singular basis and followed by the rejection of the less important set of basis from the full set of obtained basis. Therefore, there is a wastage of computational resources. For the large data sets where the expected value of $q(\ll\text{min}(m,n))$ is minimal, the waste of computational resources will increase tremendously. For large sparse complex networks, it is quite often the scenario. Therefore, deterministic SS procedure will take a long pre-processing time to provide SVD factors. In this work, we want to investigate the possible way to reduce this pre-processing time taken by SVD by introducing the random-projection (RP) based scheme to obtain the factors quickly. In this paper, we proposed a class of randomized SS procedure which can be used to omit this bottleneck. The second step (line no $3$ in the Algo-1) is essential to obtain $P$, therefore for both deterministic and randomized SS procedures, this step is required.

III Randomized subset selection scheme

The deterministic SVD is a computationally expensive method for large data sets. Furthermore, it is challenging to parallelize the standard SVD technique to utilise advanced computer architectures. Recently developed randomized algorithms based on RP [19] for low-rank approximation, are computationally efficient, accurate up-to a certain precision and robust. Therefore it outperforms the traditional matrix factorisation scheme in many practical problems [19], [20]. The randomized algorithm is powerful because of the following reasons.

• •

The scheme is computationally efficient.

• •

The main involved operations (steps) can be optimised on modern computational architecture.

The key concept of the randomized scheme is to exploit the randomness to construct a surrogate lower dimensional ($\bar{\mathcal{A}}_{q\times n}$) data matrix which captures the maximum information of a high-dimensional input data matrix $\mathcal{A}_{m\times n}$. The apparent assumption is that the input data matrix has a low-rank structure, i.e., the numerical rank is smaller than its original dimensions. Without loss of generality, we assume that $n<m$, therefore to transform from original data matrix $\mathcal{A}$ to a surrogate data matrix $\bar{\mathcal{A}}$, one has to generate a random test matrix $\Omega_{q\times m}:\Omega_{ij}\in\mathcal{N}(0,1)$ and has to operate on $\mathcal{A}$ such a way that

[TABLE]

Essentially, the operation described in Eq.8 is known as RP of the original data vectors to a lower dimensional space. Basically, operating $\Omega$ onto $\mathcal{A}$ compressed the $n$ column-vectors ($\in\mathbb{R}^{m}$) of $\mathcal{A}$ and projected them to a lower dimension space $\mathbb{R}^{q}$. Therefore, $n$ column vectors transformed from $m$ dimensional to $q$ dimensional vectors. Since, $m$ is very large and $q$ is very small, therefore this specific transformation ($\mathcal{A}\rightarrow\bar{\mathcal{A}}$) helps to reduce the computational cost as one can directly use the transformed vectors for further use. The point of concern is: how robust is the transformation? The RP can preserve the distance between each pair of the column vectors in the lower dimension within a $\epsilon$-error. The J-L lemma ensures this. Hence, the surrogate matrix $\bar{\mathcal{A}}$ can be used to obtain the approximated first $q$-right singular vectors ($\mathcal{V}$). Different approaches are also possible to obtain this. Algorithm- 2 -  5 are the representation of a class of such randomized versions.

We show the efficacy of the RP method in case of the data of the complex network. We computed the pairwise distances of the column vectors in the original and lower dimensional space for the six different real-world and model network adjacency matrices (Drosophila, Friendship, Power-Grid, US-Air, Barabasi-Albert, Erdos-Renyl Networks). The details (number of nodes, edges) of the data are mentioned in TABLE III. We are considering $q\approx 50\%\text{of the nodes}$ for complex network data. Figure 1 depicts the ratio of the pairwise distances of the vectors between the lower-dimension and original feature space. If the pair-wise distances are preserved, then the ratio should be $1$. Therefore it expected that the mean of the distribution of the pair-wise distance would be $1$ and the variance will vary depending on the dimension of the projected space and number of projected vectors. Figure 1 shows that for six different complex network data, the mean of the distribution is $1$. Therefore, the projected dimension $q=m/2$ can preserve the pair-wise distance of the column vectors in lower-dimensional space with high accuracy. Hence for these data randomized SS is applicable and the retrieval of right subspace is also possible. One should also note that for complex network data analysis, we are applying RP onto an adjacency matrix which is a symmetric matrix; therefore the compression of the column vectors or the compression of the row vectors provides the same results. Also, although in all the algorithms we denote the input matrix as a rectangular matrix $\mathcal{A}_{m\times n}$, for the complex network data they are symmetric square matrices.

The projection in low dimension is mediated by the product of a random matrix $\Omega_{q\times m}$ with $\mathcal{A}$. The low dimension projection preserves the column norms, and the computation of any decomposition (SVD/ QR) of the matrix formed by these transformed columns gets computationally cheaper as compared to performing decomposition in original higher dimensional space. We employ this trait of RP in all proposed randomized SS procedures. In the following subsections, we present the randomized subset selection versions in details.

III.1 Randomized Subset Selection 1

To reduce the computational complexity of the deterministic subset selection process, we propose the use of random projection to obtain top-$q$ right singular vectors. The original matrix $\mathcal{A}$ which can be conceived as $n$ column vectors in $m$ dimensional space, is projected onto a low dimensional space such that it becomes $\mathcal{\bar{A}}_{q\times n}$. In lower dimension, it transforms such that there are $n$ column-vectors in $q$ dimensional space, obtained by a procedure as follows,

[TABLE]

where $\Omega\in\mathcal{N}(0,1)$. Now one can compute SVD of $\bar{\mathcal{A}}$ to obtain the right singular vectors $\mathcal{V}$ of $\mathcal{A}$. Let the SVD factors of $\bar{\mathcal{A}}_{q\times m}$ are as follows

[TABLE]

The dimension of $\mathcal{V}_{\bar{\mathcal{A}}}$ is $q\times m$. Theoretically, one can think of

[TABLE]

Again,

[TABLE]

Where we have assumed that $\mathbb{E}(\Omega^{T}\Omega)=\mathbb{I}_{m\times m}$ which is the inherent property of random projection matrix due to the fact the columns are almost linearly independent as they are drawn from standard Gaussian distribution.

Using the fact from Eq.12, one can rewrite the Eq.11 as

[TABLE]

Therefore one can approximate $\mathcal{V}$ by $\mathcal{V}_{\bar{\mathcal{A}}}$ without applying SVD to the original data matrix $\mathcal{A}$. The surrogate $\bar{\mathcal{A}}$ can be used to obtain it. It is notable that only top-$q$ right singular vector of $\mathcal{V}$ can be approximated with high accuracy by $\mathcal{V}_{\bar{\mathcal{A}}}$. Other $m-q$ number of right singular vectors are not possible to retrieve. Hence this scheme can reduces the time required to compute the SVD as the number of columns are reduces by a factor $m-q$. The rest of the steps are similar as deterministic SS. The algorithm for the procedure is given in Algorithm-2.

For a system of complex network, if the network is very large, a large adjacency matrix has to be dealt with and full SVD of $\mathcal{A}$ is needed to be performed in order to find its right singular vectors $\mathcal{V}$. The above prescribed randomized SS can be useful to reduce the time complexity to obtain the SVD factors promptly.

III.2 Randomized Subset Selection 2

In step 2 of the previously prescribed randomized SS scheme (Algorithm- 2) involves SVD of a matrix of size $q\times n$; hence the time complexity of the computing SVD will be $\mathcal{O}(nq^{2})$, whereas House-holder transformed based QR decomposition [14] can reduce the time complexity further. In this sub-section, we proposed a new randomized SS scheme based on the QR-decomposition. Suppose QR-decomposition of $\bar{\mathcal{A}}_{q\times n}$ can be factorized as

[TABLE]

Here we use partial QR-decomposition as $q$ columns of ${R_{\bar{\mathcal{A}}}}_{q\times n}$ are non-singular. Previously we have shown top-$q$ columns of $\mathcal{V}$ can be approximated by $\mathcal{V}_{\bar{\mathcal{A}}}$. Therefore, our aim will be to obtain $\mathcal{V}_{\bar{\mathcal{A}}}$ from the QR-factors.

Let that SVD factors of $R_{\bar{\mathcal{A}}}$ can be written as

[TABLE]

Combining Eq:14 $\&$ 15, we can re-write

[TABLE]

From Eq.9 $\&$ 16, it is clear that $\mathcal{U}_{\bar{\mathcal{A}}}\approx Q\,\mathcal{U}_{R_{\bar{\mathcal{A}}}}$ ; $\Sigma_{\bar{\mathcal{A}}}\approx\Sigma_{R_{\bar{\mathcal{A}}}}$ ; $\mathcal{V}_{\bar{\mathcal{A}}}\approx\mathcal{V}_{R_{\bar{\mathcal{A}}}}$ Hence one can compute $\mathcal{V}_{R_{\bar{\mathcal{A}}}}$ and that can be used to approximate $\mathcal{V}_{\bar{\mathcal{A}}}$, which eventually approximates $\mathcal{V}$. These steps are written in Algorithm-3. This algorithm is not computationally efficient as it involves one QR-decomposition (step2) and another SVD factorization (step3) of a matrix of size $q\times n$. The the previously described Algorithm-2 is more computationally acceptable in comparison to this algorithm. It is further possible to discard the step3 of the algorithm by slightly modifying the algorithm which described in the next section as Algorithm-4.

III.3 Randomized Subset Selection 3

Another advancement in terms of time complexity reduction is to discard the step3 of the previously described algorithm (Algorithm-3). From the Eq.9, it is clear that

[TABLE]

Therefore, if one can compute QR-decomposition of $\bar{\mathcal{A}}^{T}$ as

[TABLE]

then from Eq.17 $\&$ 18, it is easy to relate

[TABLE]

This scheme is more efficient than previously described two different randomized algorithm (Algorithm-2, 3), as it involves only QR-decomposition (step2) of a matrix of size $n\times q$.

III.4 Randomized Subset Selection 4

This section describes the fourth randomized version of the algorithm for subset selection. The previous three versions (Algorithm-2, 3, 4) have used SVD or QR factorization of a matrix of size $q\times n$. But, if $n$ is also very large then time complexity will not be reduced to the desired level. Therefore, we are interested in transforming the data matrix to a much smaller size and retrieval of the first $q$-right singular vectors approximately. The prescribed algorithm in this section is based on constructing a matrix $Y$ of size $q\times q$ and approximates right singular vectors correctly. The transformation from $\mathcal{A}_{m\times n}\rightarrow\bar{\mathcal{A}}_{q\times n}\rightarrow{Y}_{q\times q}$ is logical and can be explained through simple algebraic relations. The first transformation ($\mathcal{A}_{m\times n}\rightarrow\bar{\mathcal{A}}_{q\times n}$), as described in the previous section, showed that the approximation of right singular vectors of $\mathcal{A}$ is possible by the right singular vectors of $\bar{\mathcal{A}}$. In this sub-section, we discuss the second transformation ($\bar{\mathcal{A}}_{q\times n}\rightarrow{Y}_{q\times q}$) and the retrieval of right singular vectors of $\bar{\mathcal{A}}$ from $Y$. From Eq.9, it is trivial to obtain,

[TABLE]

The above Eq.20 shows that $Y$ preserves the information about the left-singular vectors $\mathcal{U}_{\bar{\mathcal{A}}}$ and it reflects a eigenvalue relation of $Y$.

[TABLE]

Therefore, one can compute the eigenvalue decomposition of $Y$ and obtain the approximated left singular vectors, singular values of $\bar{\mathcal{A}}$. Suppose the eigenvalue decomposition of $Y$ is

[TABLE]

Therefore comparing Eq.21 $\&$ 22, we can get

[TABLE]

[TABLE]

The right singular vector of $\bar{\mathcal{A}}$ can be defined as

[TABLE]

III.5 Computational complexity analysis

To understand the computational complexity of proposed Algorithm 2-5 in terms of the number of floating points operations, we evaluate the complexity of required steps. We mentioned earlier that the computation of column pivoting QR-decomposition of the truncated right singular matrix is the essential step for the all randomized and deterministic SS method. Therefore, in the computational complexity analysis, we excluded that cost. All randomized algorithms involves a multiplication of the data matrix with a random projection matrix, therefore it requires $\mathcal{O}(mnq)$. Apart from that, the Algorithm 2 involves a SVD factorization of a matrix of size $q\times n$ which cost $\mathcal{O}(nq^{2})$. Similarly, the Algorithm 3, 4 involves a QR-factorization of a matrix of $q\times n$ which cost $\mathcal{O}(2nq^{2}-\frac{2}{3}q^{3})$, considering the House-Holder transformation. Whereas, the Algorithm 5 involves a eigenvalue decomposition in step3, therefore the time complexity is $\mathcal{O}(q^{3})$, which is very cheap for a low-rank structure in comparison to the other schemes.

The TABLE I. shows the detail of the complex complexity of all the described algorithms. The Algorithm 5 is more efficient in comparison to the other proposed randomized SS scheme in terms of practical applicability of the scheme. This scheme is highly parallelizable and suited for the modern computer architecture. This algorithm involves only one eigenvalue decomposition (step3) of a matrix of dimension $q\times q$. Due to the low-rank structure, the value of $q$ will be very small for a large data matrix, therefore this eigenvalue operation can be easily done on a single machine, whereas the other randomized algorithms involve a SVD/ QR-factorization of $q\times n$ sized matrix, therefore for a large $n$, single machine can not perform this operation.

If we consider the total time complexity of each of prescribed algorithm then the Algorithm4 is more cost efficient as Algorithm 5 involves a huge matrix multiplication which has a cost of $\mathcal{O}(nq^{2})$. But all the prescribed algorithms are useful for a large data matrix, therefore we can’t store the data matrix into one specific machine, hence we have to store the data in a distributed architecture. In distributed architecture, the matrix multiplication cost will be reduced drastically. Hence, cost of the algebraic operations should not be included to compare the computationally efficiency of the methods. It would be better to provide a computational complexity analysis of these algorithms in a distributed system which is beyond the scope of this paper. Therefore the comparison based on the third column of the TABLE I makes more sense as performing SVD or QR on a large dimensional data matrix is the bottleneck. Based on that comparison, all the randomized algorithm out perform the complex complexity obtain by the deterministic SS.

In the TABLE II, we show the time required to obtain the subset using all the deterministic and randomized SS methods for six networks. The values of time units are averaged over independent 20 trials. If $q<<n$, it is expected that the time required will be much less that the value in the TABLE II, which is for $q\approx n/2$.

IV Application of SS selection procedure in complex networks

For a detailed account of SS procedure application on complex networks, please refer [1]. However, for the sake of completeness, we summarise it here. The essential condition or constraint to obtaining size reduced representation of complex networks is to retain its eigenvalue spectra and to quantify any loss in matrix energy in terms of Frobenius norms difference of the $\mathcal{A}$ and the $\mathcal{A}_{1}$. On a slightly different note, the constraint conditions, i.e. preservation of full matrix energy in the subset and overlap of PSVs, may not be imposed on the SS procedure at all. For this case, one computes subset for an arbitrary value of $q$, and the subset has $q$ most linearly independent columns. In [1], we showed that even when the subset size (number of columns = $q$) was chosen to be half the size of $\mathcal{A}$, the PSV of $\mathcal{A}_{1}$ has more than 99$\%$ overlap with PSV of $\mathcal{A}$ also the loss of matrix energy was very minimal.

The amount of overlap between PSVs was quantified using Cosine Similarity (CS), which is a measure of relative orientations of the two vectors (refer eq. 24, where $\mathcal{A}^{1}$ and $\mathcal{A}_{1}^{1}$ are PSVs of main adjacency matrix and the subset). Bounded between [0, 1], CS is maximum when the two vectors are oriented along the same direction and minimum when they are perpendicular to each other. In our case, the CS is a measure of the extent of information retrieval from the main matrix into the subset.

[TABLE]

Obtaining the subset network is not very straightforward as the selected subset is generally a non-square matrix. To this end, the subset selection procedure was extended, and the rows of the selected subset were reordered using same $P$(Refer eq. 25, $\mathcal{B}$ has top $q$ rows, and $\mathcal{B}_{r}$ has remaining ones). This is intuitively justified owing to the symmetric nature of the adjacency matrix. Hence the subset network has $q$ nodes with connections defined by subset matrix $\mathcal{B}$.

[TABLE]

We find that subsets extracted through all the randomized versions of the deterministic SS procedure have excellent CS of Principal Singular Vector (PSV) with that of the original network adjacency matrix (see FIG. 2. This has two important implications. Firstly, the conservation of PSV in subsets implies that one can infer the influence of nodes from the subset PSV itself. Also, as the PSV entries are indicative of node influence (components of PSV being the eigenvalue centralities of the nodes), one can infer a node’s influence in spreading processes on the network using the subset PSV’s corresponding component. This is a good result, as doing the SVD of a large network to find PSV can be computationally expensive. Secondly, the subset network identifies the most important network structure in terms of its nodes and edges, such that selected network has enhanced information flow governing properties. We have shown the application of SS using all the deterministic and randomized SS on US Air network [21] in FIG. 3 and FIG. 4. One can see that all the deterministic and randomized algorithms can efficiently detect almost all high eigenvector centrality nodes and hence we can conclude that the subset nodes contribute maximally to the inverse participation ratio of the networks. Please refer to [1] for details. Also, we find that the loss in matrix norm of the subsets with $q\approx n/2$ is very minimal for all the network examples we took (refer TABLE III).

V Conclusion and discussion

The present manuscript presents a class of randomized subset selection procedures on complex networks data. The main highlight of this work is the use of Random Projection scheme in the process of extract top $q$ most linearly independent columns from a data matrix. The RP method thrives on rank deficiency of input data matrix. Apart from reducing the time complexity incurred due to the performing of SVD of full data matrix required for deterministic SS procedure, RP can preserve the maximum information from the data matrix. We have verified the applicability of the proposed methods on complex network datasets. The complex networks, for example, the Internet, World Wide Web and traffic network, can be huge as well as dynamically evolving. The proposed methods owing to there reduced time complexity can unburden the computing devices a lot on such datasets. Finding the spectra and eigenvectors of complex networks is of paramount importance to infer its topological and functional properties. Also, finding the most critical nodes and links or the most functional network structure is currently one of the most researched topics in complex networks. We showed that using the SS procedure, the important network structure can be extracted which captures the most influential network nodes. Using randomized version of SS, this process can be fastened by many folds. We have taken $q\approx N/2$, and showed all the results with this $q$ itself. However, $q$ can further be reduced depending on sparsity of data (small Network density) and time complexity can further be reduced. The determination of appropriate $q$ is altogether a different problem and serves as a prelude to our work. Although we have applied the randomized SS procedure to complex network data, these procedures can very well be extended to the general class of large datasets and real-time analysis of time evolving data.