Modularity-like objective function in annotated networks

TL;DR

This paper introduces a modularity-like objective function for annotated networks that balances modularity and metadata influence, enabling adjustable community detection and expanding modularity methods.

Contribution

It proposes a novel objective function that integrates metadata influence into modularity optimization, allowing adjustable community detection in annotated networks.

Findings

The objective function is a linear combination of modularity and conditional entropy.

Adjustable influence of metadata enables recovery of metadata-driven communities.

Transition exists between metadata-dominant and modularity-dominant partitions.

Abstract

We ascertain the modularity-like objective function whose optimization is equivalent to the maximum likelihood in annotated networks. We demonstrate that the modularity-like objective function is a linear combination of modularity and conditional entropy. In contrast with statistical inference methods, in our method, the influence of the metadata is adjustable; when its influence is strong enough, the metadata can be recovered. Conversely, when it is weak, the detection may correspond to another partition. Between the two, there is a transition. This paper provides a concept for expanding the scope of modularity methods.

Equations

$$\begin{array}{r}P(\mathbf{A},\mathbf{s}|\mathbf{\Theta },\mathbf{\Gamma },\mathbf{x})=P(\mathbf{A}|\mathbf{\Theta },\mathbf{s})P(\mathbf{s}|\mathbf{\Gamma },\mathbf{x})=\prod _{i<j}{p}_{ij}^{A_{ij}}(1-p_{ij}{)}^{1-A_{ij}}\prod _i{\gamma }_{s_i{x}_i},\end{array}$$

$$\begin{array}{rl}\log P(\mathbf{A},\mathbf{s}|\mathbf{\Theta },\mathbf{\Gamma },\mathbf{x})& \sim \sum _i\log {\gamma }_{s_i{x}_i}+\frac{1}{2}\sum _{ij}{A}_{ij}\log (k_i{k}_j{\theta }_{s_i{s}_j})+\frac{1}{2}\sum _{ij}\log (1-k_i{k}_j{\theta }_{s_i{s}_j})\\ & \sim \sum _x\sum _s{N}_{sx}\log \frac{N_{sx}}{N_x}+\frac{1}{2}\sum _{ij}{A}_{ij}\log {\theta }_{s_i{s}_j}-\frac{1}{2}\sum _{ij}{k}_i{k}_j{\theta }_{s_i{s}_j},\end{array}$$

$$\sum _x\sum _s{N}_{sx}\log \frac{N_{sx}}{N_x}=N\sum _x\sum _{s}p(s,x)\log \frac{p(s,x)}{p(x)}=N\sum _{x}p(x)\left(\sum _{s}p(s|x)\log p(s|x)\right)=-NH(S|X),$$

$$\theta_{st}=\left\{\begin{array}[]{rcccl}&\theta_{in}&&\text{if}&{s=t}\\ &\theta_{out}&&\text{if}&{s\neq t}\end{array}\right..$$

$${\theta }_{st}=({\theta }_{in}-{\theta }_{out}){\delta }_{st}+{\theta }_{out},$$

$$\log {\theta }_{st}=(\log {\theta }_{in}-\log {\theta }_{out}){\delta }_{st}+\log {\theta }_{out}.$$

$$\begin{array}{rl}\frac{1}{2}\sum _{ij}{A}_{ij}\log {\theta }_{s_i{s}_j}-\frac{1}{2}\sum _{ij}{k}_i{k}_j{\theta }_{s_i{s}_j}& \sim M\log \frac{{\theta }_{in}}{{\theta }_{out}}\frac{1}{2M}\sum _{ij}\left(A_{ij}-\frac{2M({\theta }_{in}-{\theta }_{out})}{(\log {\theta }_{in}-\log {\theta }_{out})}\frac{k_i{k}_j}{2M}\right){\delta }_{s_i{s}_j},\end{array}$$

$$\frac{1}{2M}\sum _{ij}\left(A_{ij}-\gamma \frac{k_i{k}_j}{2M}\right){\delta }_{s_i{s}_j}-\alpha H(S|X)=Q(\gamma )-\alpha H,$$

$$\mathbf{\omega}=\frac{4c}{N(1+\epsilon_{1})(1+\epsilon_{2})}\left(\begin{array}[]{cccc}1&\epsilon_{2}&\epsilon_{1}&\epsilon_{1}\epsilon_{2}\\ \epsilon_{2}&1&\epsilon_{1}\epsilon_{2}&\epsilon_{1}\\ \epsilon_{1}&\epsilon_{1}\epsilon_{2}&1&\epsilon_{2}\\ \epsilon_{1}\epsilon_{2}&\epsilon_{1}&\epsilon_{2}&1\end{array}\right),$$

Full text

I Introduction

Community structure, a partition of nodes in which the density of edges within groups is denser than that between groups, is an important large-scale structure in complex networks, and has attracted significant attention in recent years Fortunato_review ; Newman_review ; Fortunato_review_16 . Many methods have been proposed for detecting community structure. Here, we focus on two: statistical inference Decelle_SatInf ; Decelle_SatInf2 ; Karrer_DCSBM and modularity-based methods Newman_modularity . Statistical inference is flexible; it can be used for different purposes, such as detecting generalized communities Newman_Gen or estimating group number Newman_qnum . Additionally, statistical inference can be used for detecting annotated networks, in which annotations or metadata that describe the attributes of nodes (such as the age, gender, or ethnicity of individuals in a social network) accompany the network structure Newman_metadata . Newman-Girvan modularity Newman_modularity is the most popular measure of the quality of a partition. Several modifications have been proposed for measuring different unannotated network structures, including weighted Weighted_mod , directed Directed_mod , bipartite Bipartite_mod and multiplex networks Multiplex_mod . However, modularity in annotated networks has not been defined. In the paper, we focus on the objective function in these networks and its relation to Newman-Girvan modularity.

The equivalence between modularity optimization and maximum likelihood Zhang_MBP ; Newman_equivalent may inspire us to our goal. However, this derivation is for unannotated networks. In the statistical inference method, the model of a network with community structure is defined and then fit to observed network data. In most cases, the model parameters are estimated by likelihood maximization; for different considerations or data types, the likelihoods are different. The likelihood in annotated networks differs from (though is similar to) that of unannotated networks. Herein, we ascertain the modularity-like objective function whose optimization is equivalent to the maximum likelihood in annotated networks. We demonstrate that the modularity-like objective function is a linear combination of modularity and conditional entropy. In contrast with the statistical inference method, we set a variable parameter that controls the influence of the metadata. Our results, in both synthetic and real-world networks, demonstrate that if the parameter is strong enough, the metadata can be recovered; however, if it is weak, our method may recover another partition that is more evident, instead of the metadata. Between the two, we find a transition from the more evident partition to the metadata.

II Method

To illuminate our method, we first provide a brief introduction to the likelihood of statistical inference in annotated networks Newman_metadata . In this paper, we consider only the case in which the metadata is a classification or a partition of nodes, $\mathbf{x}=\{x_{i}\}$. In this method, a degree-corrected stochastic block model is defined to a network. The probability, or likelihood, that the model generates a particular network $\mathbf{A}$ and group assignment $\mathbf{s}$ with $q$ groups is

[TABLE]

where $\gamma_{sx}$ is the probability that a node is assigned to group $s$ given its metadata $x$; $\mathbf{\Gamma}$ denotes the matrix of parameters $\gamma_{sx}$; $p_{ij}=k_{i}k_{j}\theta_{s_{i}s_{j}}$ is the probability of node $i$ connecting to $j$, where $k_{i}$ ($k_{j}$) is degree of node $i$ ($j$) and $\theta_{st}$ are parameters indicate the strength of connection between groups; and $\mathbf{\Theta}$ denotes the matrix of parameters $\theta_{st}$.

The likelihood maximization is equivalent to the maximization of the logarithm

[TABLE]

where $N_{sx}$ is the number of nodes assigned to group $s$ with annotation $x$ and $N_{x}$ is the number of nodes with annotation $x$. The first term is:

[TABLE]

where $N$ is the number of nodes in the network and $H(S|X)$ is the conditional entropy. The second and third terms induce the modularity Newman_equivalent . The planted partition model Condon_planted is a special case of the stochastic block model in which the parameters $\theta_{st}$ describing the community structure take only two different values:

[TABLE]

Eq. (4) implies that

[TABLE]

[TABLE]

Thus, the second and third terms of Eq. (2) are Newman_equivalent

[TABLE]

in which some constants have been dropped. The maximization of Eq. (2) is equivalent to the maximization of

[TABLE]

where $\gamma=\frac{2M(\theta_{in}-\theta_{out})}{(\log\theta_{in}-\log\theta_{out})}$ and $\alpha=\frac{N}{M(\log\theta_{in}-\log\theta_{out})}$, which can be estimated. In this paper, we set $\gamma=1$ and treat $\alpha$ as a variable parameter to control the balance between the structure and metadata. High values of $\alpha$ drag the result to the metadata, though the principle for selecting the appropriate value of $\alpha$ is still unknown. We emphasize that our goal is to determine how metadata can be recovered, so the number of groups of detected partitions is equals to that of the metadata in most case. Eq. (8) is the modularity-like objective function, which is a linear combination of modularity and conditional entropy. We have demonstrated that the optimization of Eq. (8) is equivalent to the maximum likelihood of Eq. (1). As the modularity-like objective function is known, we use simulated annealing Guimera_simuanneal for optimization with a fixed $q$.

III Results

Our first example is a network generated by a stochastic block model (SBM). In SBM, nodes are randomly assigned to one of $q$ groups and the probability that any pair of nodes connects depends on the node memberships, $p_{ij}=\omega_{s_{i}s_{j}}$. In this case, we set $q=4$ and

[TABLE]

where $c$ is the average degree in the network. It is also a special case of a nested SBM Peixoto_hierarchical , in which $L$ (here $L=2$) community structures are coupled. In the first partition $\mathbf{s}$, original groups 1 and 2 are merged into one group, and the remaining two original groups are merged into another group. In partition $\mathbf{s^{\prime}}$, original groups 1 and 3 are merged and the left original groups are merged. $\epsilon_{1}$ and $\epsilon_{2}$ denote the strength of the two planted structures. In this case, we set $N=2000$, $c=3$, $\epsilon_{1}=0.1$, $\epsilon_{2}=0.15$ and metadata $\mathbf{x}=\mathbf{s^{\prime}}$. $\mathbf{s^{\prime}}$ is much weaker than $\mathbf{s}$, so that with this metadata, the method in Newman_metadata recovers $\mathbf{s}$ rather than $\mathbf{s^{\prime}}$. However, by adjusting the influence of the metadata with parameter $\alpha$, our method can recover $\mathbf{s}$ in an appropriate range (see Fig. 1).

Fig. 1 shows that the modularity-link function looks like a broken line with three segments. There is a transition at $\alpha_{c}=0.052$. Below this transition, $\alpha$ is small enough that structure plays a leading role. Optimization of the objective function finds the partition with the highest modularity. In this case, $Q(\mathbf{s})>Q(\mathbf{s^{\prime}})$, so $\mathbf{s}$ is recovered. Above $\alpha_{c}$, the value of overlaps (i.e., the fraction of nodes correctly detected) with the two structures exchanges. If $\alpha$ is not high, both the structure and metadata play important roles in detection. The metadata provides all information of $\mathbf{s^{\prime}}$, $H(\mathbf{s^{\prime}}|\mathbf{x})=0$; while it provides no information to $\mathbf{s}$, $H(\mathbf{s}|\mathbf{x})$ is high. Thus, the metadata drags the detection to it. However, the landscape has a smooth valley surrounding $\mathbf{s^{\prime}}$ Good_Landscape . Due to fluctuation, there are some partitions that are correlated with $\mathbf{s^{\prime}}$ (i.e., the Hamming distance to $\mathbf{s^{\prime}}$ is low) with higher modularity-like objective functions than those of $\mathbf{s^{\prime}}$. Optimization methods will recover one of them, so the overlap between the detected partition and metadata is high but not equal to 1. Only when $\alpha$ is high enough, metadata plays crucial role and can be recovered absolutely.

Our second example is a network generated by a planted partition model, which is a special case of SBM with edge probabilities $p_{in}$ and $p_{out}$ for within-group and between-group edges. We generated node metadata that matched the true planted assignments, but with an error rate of $\rho=0.2$ to indicate random noise. Without metadata, or if $\alpha=0$, the approximate planted structure can be recovered. As $\alpha$ increases, detection is gradually dragged to the metadata (see Fig. 2). If $\alpha$ is high enough, the metadata is recovered absolutely and the overlap with the planted structure was $1-\rho$. The transition in Fig. 2 is not as strong as that in Fig. 1; the overlap with the planted structure in Fig. 2 changes continuously at $\alpha_{c}$. The planted structure was recovered best at an $\alpha$ value of about $0.34$. In Newman_metadata , the strength of the metadata is fixed and may be not the best choice.

Our third example is a network of students drawn from the US National Longitudinal Study of Adolescent to Adult Health AddHeal . This network consists of a high school (US grades 9 to 12) and its feeder middle school (grades 7 and 8). The annotations of high/middle school and ethnicity construct two possible partitions (see Fig. 3(a) and (b)). Between the two, the school is more evident than ethnicity; thus, we treat ethnicity as the metadata. The ethnicity annotation is so weak that with this metadata, the method in Newman_metadata recovers the school level rather than ethnicity. However, with $\alpha$, our method can recover ethnicity in an appropriate range (see Fig. 3(d)-(f) and Fig. 4). Here, we use the normalized mutual information (NMI) Danon_NMI rather than overlap to measure how the detected partition matches the annotation, because the detection may have a different group number than the annotations.

IV Conclusion and discussion

In this paper, we ascertain the modularity-like objective function whose optimization is equivalent to the maximum likelihood in annotated networks. We demonstrate that the modularity-like objective function is a linear combination of modularity and conditional entropy, with a variable scale $\alpha$ that indicates the influence of the metadata. Unlike in the statistical inference method, our method allows us to adjust the influence of the metadata. Examples in synthetic and real-world networks show that for an appropriate range of $\alpha$ (in which the influence is sufficiently strong), the metadata can be recovered. However, when $\alpha$ is low, another partition may be detected. Between the two values, there is a transition phase.

The statistical inference method is flexible, and it can be used to detect generalized communities Newman_Gen and estimate group number Newman_qnum . It is therefore interesting to find the corresponding modularity-like objective functions. In this paper, we optimized the modularity-like objective function by simulated annealing. Other optimization algorithms, such as belief propagation Zhang_MBP , are left for future work.

Acknowledgments

This work is funded by the NSFC (Grant Nos. 11275186, 91024026 and FOM2014OF001).