Compressing networks with super nodes

TL;DR

This paper introduces a method to compress large networks into smaller 'super node' networks to improve the efficiency and stability of community detection algorithms, while maintaining accuracy.

Contribution

The paper presents a novel network compression technique using super nodes based on CoreHD ranking, enhancing community detection speed and stability.

Findings

Community detection on super node networks is significantly faster.

Partitions on compressed networks are more stable across runs.

Results closely match those from full network analysis.

Abstract

Community detection is a commonly used technique for identifying groups in a network based on similarities in connectivity patterns. To facilitate community detection in large networks, we recast the network to be partitioned into a smaller network of 'super nodes', each super node comprising one or more nodes in the original network. To define the seeds of our super nodes, we apply the 'CoreHD' ranking from dismantling and decycling. We test our approach through the analysis of two common methods for community detection: modularity maximization with the Louvain algorithm and maximum likelihood optimization for fitting a stochastic block model. Our results highlight that applying community detection to the compressed network of super nodes is significantly faster while successfully producing partitions that are more aligned with the local network connectivity, more stable across multiple (stochastic) runs within and between community detection algorithms, and overlap well with the results obtained using the full network.

Tables

Table 1. Table 1: Network data characteristics.

Dataset (* indicates subgraphs)	# Nodes	# Edges
As22* (Internet)	22,801	48,270
Enron*	32,374	178,195
CMatter* (Condensed matter 2003 collab.)	17,816	83,337
Dblp* (com-DBLP)	150,801	639,330
Amazon* (com-Amazon)	77,463	209,887
Email (email-EuAll)	265,214	420,045
Stanford (web-Stanford)	281,903	2,312,497
Notre Dame (web-Notre Dame)	325,729	1,497,134
BrightKite (loc-BrightKite)	58,228	214,078

Equations

$$Q=\frac{1}{2M}\sum _{i,j}[a_{ij}-\gamma \frac{k_i{k}_j}{2M}]\delta (z_i,z_j),$$

$$p(\mathbf{a}\mid \mathbf{z},\mathbf{\pi })=\prod _{i\ne j}^N\prod _{k,l}^K({\pi }_{kl}^{a_{ij}}(1-{\pi }_{kl}^{1-a_{ij}}){)}^{\delta (z_i=k,z_j=l)},$$

Full text

1 Introduction

Networks appear across disciplines as natural data structures for modeling relations between entities, such as regulatory interactions between genes and proteins, and social connections between people. In practice, most networks are large and have intricate substructures that can be interrogated for further insights about the underlying data. A common practice for extracting subgraphs of interest is community detection [1, 2, 3], which aims to partition network nodes into groups based on the group-level connectivity patterns. For example, assortative communities are typically defined through some measure of a greater weight of within-group edges compared to those between groups. Communities can be identified through various optimization problems, including likelihood maximization [4, 5, 6] and quality function optimization [7, 8, 9], or from spectral properties of associated matrices [10].

In this paper, we specifically examine how the compressed version of a network can be handled by modularity maximization and likelihood maximization. To maximize modularity in a network with $N$ nodes, one seeks to find the node-to-community assignment vector, ${\bf z}=[z_{1},z_{2},\dots z_{N}]$ that maximizes modularity, $Q$, defined by

[TABLE]

where $a_{ij}$ encodes presence and weight of a possible edge between $i$ and $j$, $k_{i}=\sum_{j}a_{ij}$ gives the strength of node $i$, $\delta(z_{i},z_{j})=1$ if nodes $i$ and $j$ have the same community assignment and 0 otherwise, $M$ denotes the number of edges and $\gamma$ is the resolution parameter controlling community sizes. Alternatively, to find community structure maximizing likelihood, one can fit a stochastic block model (SBM), which assumes that connectivity patterns of $N$ nodes in a network with $K$ communities can be modeled according to their community memberships through a probability matrix ${\bm{\pi}}$, where ${\pi_{kl}}$ gives the edge connection probability between communities $k$ and $l$. Assuming no corrections due to node degrees (degree-corrected versions also exist), one seeks to $(\bf z)$ maximize,

[TABLE]

where $\delta({z_{i}=k,z_{j}=l})$ is 1 if $z_{i}=k$ and $z_{j}=l$, and 0 otherwise. One can maximize this objective with the expectation maximization (EM) algorithm, belief propagation, and efficiently with an iterative approach which, similar to Louvain, agglomerates blocks of nodes at each iteration [6]. Specifically, at each iteration a Metropolis-Hastings accept-reject sampling approach is used to compute a probability of merging blocks based on how the merge affects the likelihood. These agglomerative heuristics simplify a computationally challenging task but can still be time consuming for large networks and often give rise to large variability in returned partitions across multiple runs of the algorithms. We seek to explore how a compressed network representation can improve some of these issues.

The identification of communities in networks is in some ways similar to multi-label image segmentation, which aims to partition a grid of pixels into contiguous regions corresponding to objects in the image. In this sense, each segmented region can be viewed as a community [11]. To speed up segmentation for large images, a popular approach is to avoid computing segmentations at the pixel level and instead reformulate the segmentation problem based on larger-scale image primitives that are likely part of the same partition. Specifically, this can be accomplished by super pixels that aggregate pixels together in a way that faithfully adheres to image boundaries, maintaining or improving segmentation accuracy [12]. The SLIC super pixel method [12] chooses seed pixels across the image’s pixel grid to serve as the super pixel centers and then iteratively grows out and recomputes based on aggregation with neighboring pixels with similar visual features. Motivated by super pixels, we wish to define seed nodes in networks that can be used as a starting point to grow out ‘super nodes’ to define a new, smaller network upon which we apply standard community detection algorithms. Creating a direct analog of super pixels in networks is challenging because the inherent geometry of a network can be quite different from the grid layout of an image (where simple neighborhood structures such as 4- or 8-neighborhoods are typically used), and we need ensure seeds are well distributed across the network. Further, while super pixels are largely constrained by the structure of the pixel grid (i.e. proximity between pixel pairs matter), their definition also incorporates extra image features to refine members of a super pixel set, whereas in network community detection we typically only have the edges of the network to work with. Finally, the performance of a super pixel representation of an image can be objectively validated from the quality of the corresponding segmentation result, with reference to human-specified objects in images; in contrast, community detection is typically an unsupervised, exploratory data analysis technique with limited available notions of ‘ground truth’ [13]. As such, we must develop measures that can be used to validate the quality of the super node representation.

Our approach in defining a smaller network of super nodes is a form of network compression. A useful approach developed previously is the ‘KeepAll’ method [14], which seeks to prioritize a set of nodes and retain only the smallest set of additional nodes required for the induced subgraph of prioritized nodes to be connected. Two other network compression methods related to our objective in defining super nodes are explored in Refs. [15, 16]; but these approaches differ from our proposal in that they seek to define super nodes along with additional side information about relationships between node pairs. Ref. [15] describes ‘super genomic network compression’ to reduce the number of edges in a large protein interaction network. To do this, the authors identify ‘clusters of orthologous groups’ of proteins, or proteins that give rise to similar functions in different species and originated from a common ancestor. Members of an orthologous group are connected as a star network, with the center node as one member of the orthologous group. Furthermore, edges between orthologous groups are replaced by a single weighted link reflecting the pairwise group evolutionary similarity. Ref. [16] defines super nodes by defining ‘must link’ and ‘cannot link’ constraints between pairs of nodes, agglomerating as many nodes as possible sharing must link constraints while being cautious about agglomerating nodes that cannot link.

To define the super node representation of an $N$-node network, we first select $S\ll N$ seed nodes, agglomerate the remaining $N-S$ nodes with the seeds to create super nodes, and specify the network between these super nodes. Community detection can then be applied to the $S$-node network representation. Figure 1 visualizes this approach, with details provided in Sec. 2.

The aims of this work are twofold: First, we seek an effective way to define a super node representation of a network that can then be used in standard community detection algorithms. Second, we wish to establish a principled approach to validate the quality of a super node representation. Our approach to defining super nodes aims to accomplish the following.

Decreased runtime for community detection: Even though recently developed heuristics for maximizing modularity [7] and fitting SBMs [6] are highly efficient relative to previous approaches for performing the same computational optimizations, these methods can still be time consuming for large networks. We aim to reduce runtime for large networks, moving most of the computational cost in practice from tasks scaling with the size of the network to alternatives scaling with the (much smaller) size of the super node representation. 2. 2.

Decreased stochastic variability of community detection algorithm output: In large networks, there is often significant variability across multiple runs of the same algorithm (employing computational heuristics to solve NP-Complete optimizations), as well as differences between various community detection algorithms. We expect applying community detection to a well-chosen super node representation to decrease the observed variability. 3. 3.

High local agreement: In defining super nodes, we inherently assume that the identified communities should agree with the local network connectivity in that members of a neighborhood should be more likely to have the same community assignment, provided that the super nodes were constructed to minimize aggregation across community boundaries. 4. 4.

Consistent with communities found using the full network: The identified community structure should be relatively similar to the distributions of results that would have been obtained through applying community detection to the full network.

Organization. In section 2, we describe the specific algorithms we employ for choosing seeds, growing these out to super nodes, and specifying the network of super nodes. Our method of selecting seed nodes is a novel application of the CoreHD algorithm [17], previously used for network decycling and dismantling. Section 3 presents experiments used to understand and quantify the quality and usefulness of the super node representation, addressing the four objectives listed above. Finally, in section 4, we discuss other possible applications of super nodes and directions for future work.

2 Method for Generating a Super Node Representation of a Network

We create a super node representation through three steps, outlined in Figure 1. First, we define seeds. Next, we ‘grow’ super nodes by assigning the remaining nodes to seeds. Finally, the network of super nodes is defined by agglomerating edges, and can then be used in a community detection algorithm.

2.1 Identify Seeds in the Network

To define $S$ seeds, we aim to identify a set of nodes $S^{*}$ that are individually central to the network and to their communities, and that are well separated from one another. Such problems are related to influence maximization, where one identifies a small number of nodes in the network from which to effectively spread influence or diffuse messages across the network [18, 19]. The most naive approach is to select nodes with highest degree, and this might be perfectly reasonable under various circumstances. Importantly, the selection of nodes with highest degree is computationally fast, requiring $O(M)$ operations, summing over the $M$ edges to calculate the degrees of the $N$ nodes.

At slightly higher computational cost, we employ the CoreHD algorithm, which nearly optimally identifies nodes in network decycling and dismantling [17]. CoreHD recursively identifies the highest degree node in the 2-core (the maximal connected subgraph in which all nodes have degree at least 2 within the subgraph, obtained efficiently by recursively pruning away all nodes of degree less than 2). At each iteration, the removed node is added to our seed set and the now smaller 2-core and their degrees are recomputed. The difference between selecting highest degree nodes and CoreHD for our present task may be small, both in terms of result and computational cost. In particular, because we will only select $S\ll N$ seeds, there is reduced opportunity for the removals to lead to subgraphs with substantial differences between degree order in the graph and its 2-core [cf. selection of $O(N)$ removals in network dismantling]. Indeed, in our experience, simply selecting the highest degree nodes as the seeds often works well in practice. Because of the minimal extra computational cost for computing the 2-core, we use CoreHD for all of our results shown here. In Figure 2, we plot a sample network generated from the stochastic block model, with nodes colored according to community assignments and black nodes indicating seeds, showing the seeds are distributed across all 8 communities. In supplementary figures 2 and 3, we compare the communities obtained selecting seeds by different methods with the full network communities.

2.2 Grow Super Nodes Around Seeds

Once the set of seeds, $S^{*}$, is defined, we ‘grow’ them out agglomerating nearby nodes to build the super nodes. We formally define a super node as a subset of one or more nodes from the original network, ${\mathcal{X}}$. To do this, seeds are grown out to engulf nodes in increasing neighborhood orders until either all nodes are assigned to a super node center or until a user-defined number of neighborhood orders has been considered. The maximum order, $o_{max}$, can be specified to control the maximum order neighborhood to consider in building the super nodes. If after $o_{max}$, there are still unassigned nodes, the unassigned nodes are not used to build the new super node network and are all ultimately assigned to the same periphery community as they are not considered relevant to the network core. Depending on the number of chosen super nodes, $S$, the degree distribution of the original network and the quality of the chosen seeds at collapsing the network, different networks will require repeating the agglomeration process for different neighborhood orders if one wants to ensure every node is assigned to a super node. The output is a vector, ${\bf s}$ of length $N$, which gives the node-to-supernode assignments for the nodes in the original network.

2.3 Create Network of Super Nodes

Finally, after growing the super nodes, we must create a new network representation of the super nodes. To do this, we create a weighted network, $\mathcal{W}$, where each super node is a node and the weight of the edge between a pair of distinct super nodes is the total weight of edges in the original network ${\mathcal{X}}$ between pairs of nodes assigned to the respective super nodes. For pairs of super nodes whose members have no edges between them in ${\mathcal{X}}$, there is no corresponding edge in ${\mathcal{W}}$. By definition, we construct ${\mathcal{W}}$ with no self loops.

After applying community detection to the super nodes, their community assignments are mapped back to their constituent $N$ nodes of the original network, ${\mathcal{X}}$. In section 3, we use this final $N$-length matched community assignment, which we denote as ${\bf z}$. We consider the node-to-community assignment ${\bf z}^{Full}$ obtained by applying community detection to the full network and the mapped result ${\bf z}^{SN}$ obtained by applying community detection to the super node representation.

3 Results

To demonstrate the effectiveness of super nodes, we performed experiments to analyze the runtime, partition variability, alignment of communities with local network connectivity, and agreement with the communities obtained from the full network. We considered 9 network data sets (see Table 1) from the Stanford Network Analysis Project database111https://snap.stanford.edu/data (Enron, Amazon, Dblp, Email, BrightKite,

Stanford, Notre Dame) and Newman’s collection222http://www-personal.umich.edu/~mejn/netdata/ (As22, CMatter). We treat all networks as undirected. To explore a range of network sizes, in some of these cases we used large subgraphs defined by the union of all nodes of degree $\geq 2$, their neighbors, and next nearest neighbors. We use the Louvain algorithm333as implemented in https://github.com/vtraag/louvain-igraph for modularity maximization [7] and the stochastic block model (SBM) inference444as implemented in https://graph-tool.skewed.de/ described in [6]. In all experiments, we created the super node representation with $S=500$ super nodes.

3.1 Objectively Comparing Partitions on Possibly Different Scales

A challenge in directly comparing the community partitions on the full and super node network representations is the difference in scales between the partitions. For example, using the full network typically produces significantly more communities than under the super node representation. In an attempt to compare community partitions with similar size distributions in the subsequent experiments, we can adapt the scales obtained from the Louvain algorithm and SBM fitting.

In community detection with the Louvain algorithm, we identified comparable resolution parameters (controlling community size) to apply to the full network that would produce a size distribution agreeing as much as possible with the community partition in the super node network. Further, we compute all following experiments using both the default resolution parameter and the ‘matched’ parameter. While the default resolution parameter is $\gamma=1$, in our analyses we computed partitions of the full network using several different $\gamma\in[0.05,2.5]$. To choose the matched resolution parameter on the full network, we first find the community partition using the super node representation. For each partition, we then rank nodes based on the sizes of the communities to which they belong. For each partition of the full network (at different resolution parameters), we then consider the similarity of this ranking with that from the super node communities, measuring this similarity by Kendall’s tau correlation. We identify the resolution parameter producing the highest Kendall’s tau correlation, referring to this resolution parameter as the ‘matched parameter’ in the remainder of the text, while we refer to the standard $\gamma=1$ as the ‘default’ resolution parameter.

In fitting SBMs, we chose to fit a model with the same number of blocks that was found in the super node representation using the standard optimization and model selection strategies discussed in [6]. We refer to the ‘matched’ version as that using the number of blocks identified by the model selection on the super node representation, while the ‘default’ result is obtained using the model selection strategy on the full network. In subsequent experiments, we compare both the ‘matched’ and ‘default’ versions to ensure our results are not artificially influenced by the scale of the community sizes.

3.2 Experiments

For each of the 9 networks, we recorded the runtime required to identify communities with the Louvain algorithm and stochastic block model inference procedure under the full and super node representations (Figure 3). The Louvain algorithm is fast and scales well, at $O(M)$ per iteration for $M$ edges, with its relative speed and high modularity values contributing to its popularity. While the reported runtimes may seem quite modest, in practice it is common to run many realizations of the algorithm (hundreds, thousands, or even more for large networks) to explore resolution parameters and stochastic variation due to node order in the heuristic. We note a large increase in runtime for the full Stanford network, with over 2 million edges. As also observed in the figure, fitting a stochastic block model, at $O(N{\ln}^{2}N)$ for sparse networks in this implementation [6], becomes significantly slower on the full networks with more than 200,000 edges.

While we see a significant improvement in community detection runtime from using super nodes for both methods, the benefit in the SBM fitting is particularly large, especially for the bigger networks (DBLP, Stanford, Email). In moving to the super node representation, we traded out large-coefficient scaling-with-$N$ community detection computations for those scaling with $S\ll N$ (with possible increases due to the increased density of the super node representation), at the cost of constructing the super node representation. In particular, we observe the SBM runtimes on super nodes appear to be relatively independent of $M$. We note that each of the three steps building our super-node representation is $O(M)$, so in the large graph limit the expected gain of our approach may be only a constant factor over Louvain iterations and the SBM fitting (up to logarithmic factors). In the present calculations, we have not endeavored to optimize the runtime to build our super node representations; even so, the three steps building the $S=500$ super node representation of the Stanford network in our current implementation together take $\sim 350$ sec with CoreHD and $\sim 200$ sec using highest degree nodes. While this alone might not seem like a large improvement compared to a single realization of running Louvain or fitting an SBM, the computational gains compared to generating multiple community partitions can be quite significant.

Next, we used normalized mutual information (NMI) [20] to study similarities in partitions within and between community detection algorithms and between the full and super node representations. With the Louvain algorithm and with fitting SBMs according to the inference described in Ref. [6], sampling and non-convexity in the optimization landscape lead to large variability in partitions. In Figure 4 we computed normalized mutual information (NMI) between all pairs of partitions within and between multiple runs of Louvain and SBM fitting. A high normalized mutual information between a pair of partitions indicates that the algorithms identified similar community structures. The distribution of these pairwise NMI values for the full network and super node representation are visualized with box plots in Figure 4A and B, respectively. For each network, the purple, orange and maroon boxes show the distribution of pairwise NMIs between 10 runs of the Louvain, SBM fitting, and all individual runs of Louvain against all runs of SBM fitting, respectively. The most significant improvement we observe under the super node representation is between runs of the Louvain algorithm and SBM fitting, suggesting that the new compressed representation of the network has prominent structural features that are robustly identified with both approaches. The Louvain algorithm is generally less variable than fitting an SBM, but we also observed decreased variability in the fitting of stochastic block models on the super node representation.

While we see a reduction in variability in partitions on the super node representation, we also quantified the similarity in partitions on the full network under default parameters to those on the super node network by computing the NMI between their respective partitions, NMI$({\bf z}^{Full},{\bf z}^{SN})$. In Figure 4, we indicated the mean pairwise NMI between a full network and 5 super node partitions (again with $S=500$ super nodes) with a star. That is, a star in the Louvain (SBM) column indicates the mean pairwise NMI computed from NMI$({\bf z}^{Full},{\bf z}^{SN})$ using Louvain (SBM). As previously mentioned, a challenge in objective evaluation of partition similarity between representations is the difference in community size distributions. As a result, it is quite difficult to achieve a high NMI using default parameters. However, even these NMIs indicate structural similarity between partitions. Given the high variability within and between community detection algorithms indicated by moderate-to-low NMI values, one does not lose significantly more information by simply using the compressed super node network representation, even without matching scales. While these experiments are performed with $S=500$, we study the effects from using other $S$ values in supplementary tables 1 and 2.

While we have emphasized benefits in the mechanics and usability of running standard community detection algorithms, we now seek to address whether the communities that we find using the super node representation align with local network connectivity so that neighbors are more likely to have similar community assignments and how this alignment compares with what we would have found by community detection on the full network. While we visualize this qualitatively for the As22 network in Figure 5E-H, we also designed a prediction task to quantify this alignment. In this prediction task, we seek to take a node-to-community partition (from either the full or super node network representations ${\bf z}^{Full}$ or ${\bf z}^{SN}$, respectively) and the full network ${\mathcal{X}}$ to see how accurately we can predict members of a community for different neighborhood sizes. For network ${\mathcal{X}}$ with node-to-community assignments ${\bf z}$, we assign a probability distribution to each node over all of the communities under ${\bf z}$. For a neighborhood order $o$ (x-axis in Figure 5), we say that node $i$ has probability of being in community $k$, based on what fraction of its neighbors belong to that community under ${\bf z}$. Then for each community in ${\bf z}$, we perform a binary prediction task for whether each node of ${\mathcal{X}}$ should be assigned to that community, according to the computed probability distributions for all nodes with respect to that community.

We sweep the probability parameter, $p$, representing the required threshold probability for a node to achieve in order to be assigned to a community in this binary classification task. By sweeping $p$ for each of the communities, we compute an ROC curve for each community and the corresponding areas under the curve (AUC). Finally, we use the minimum AUC value as our summary statistic of this task, with a high AUC value indicating that the neighboring regions of a node were strong predictors of community assignments, as shown in Figure 5A-D. All experiments are performed on 5 networks (As22, Enron, CMatter, Dblp, Amazon) and for both the matched and default parameters (indicated by line type) in the full network. (Recall from section 3.1 that the matched parameters for the full network were chosen based on the super node partition results under default settings; hence, there is no corresponding ‘matched’ set for the super nodes in these plots.) We observe in most cases using the super node representation improves the minimum AUC value, indicating that communities obtained from this representation have higher agreement with local connectivity by this measure.

To qualitatively evaluate how the super node representation is able to ultimately partition the network into communities that are locally relevant, we visualize the As22 network, with nodes colored by communities identified under the full network (E,G) and super node representation (F,H) under Louvain (E,F) and SBM (G,H). Consistent with the quantitative results in Figure 5A-D, we observe that the super node representation leads to an effective coarse-graining, with the community labels appearing to be qualitatively consistent across large regions of the network.

4 Conclusion and Future Work

We developed an approach for compressing a network into a super node representation that can be used in standard community detection algorithms. Using the smaller super node network reduces runtime and the variability between multiple runs of the same community detection algorithm. Our results also demonstrate that the communities in the super node network are better aligned with local network neighborhoods in a predictive sense, while still being in relatively good alignment with the partitions obtained using the full network.

Super nodes may be useful in a variety of contexts where large datasets are otherwise difficult to mine and interpret. For example, one might visualize the super node version of the network rather than the entire network, or use the members of a super node to identify redundant information in the network. Future work on super node representations could include the extension of this method to directed, signed, attributed, or bipartite networks. Additionally, one might consider a probabilistic model framework, attempting to infer latent super node assignments.