Detecting Local Community Structures in Social Networks Using Concept Interestingness

TL;DR

This paper introduces COIN, a novel community detection method in social networks that uses concept lattice construction and formal concept analysis to identify local community structures more accurately and efficiently.

Contribution

The paper presents COIN, a new community detection approach leveraging concept interestingness and formal concept analysis, outperforming existing algorithms in accuracy and speed.

Findings

COIN detects communities more accurately than Edge betweenness, Fast greedy modularity, and Infomap.

COIN effectively extracts cliques and bridges in social networks.

The method quickly identifies local community structures in real-world social networks.

Abstract

One key challenge in Social Network Analysis is to design an efficient and accurate community detection procedure as a means to discover intrinsic structures and extract relevant information. In this paper, we introduce a novel strategy called (COIN), which exploits COncept INterestingness measures to detect communities based on the concept lattice construction of the network. Thus, unlike off-the-shelf community detection algorithms, COIN leverages relevant conceptual characteristics inherited from Formal Concept Analysis to discover substantial local structures. On the first stage of COIN, we extract the formal concepts that capture all the cliques and bridges in the social network. On the second stage, we use the stability index to remove noisy bridges between communities and then percolate relevant adjacent cliques. Our experiments on several real-world social networks show that COIN can quickly detect communities more accurately than existing prominent algorithms such as Edge betweenness, Fast greedy modularity, and Infomap.

Tables

Table 1. Table 1: The formal context 𝕂 ~ ~ 𝕂 \tilde{\mathbb{K}} for the network of Figure  1 .

$𝒢$	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1	1	1	1	1	0	0	0	0	0	0	0	0	0	0	0
2	1	1	1	1	0	0	0	0	0	0	0	0	0	0	0
3	1	1	1	1	0	0	0	0	0	0	0	0	0	0	0
4	1	1	1	1	1	0	0	0	0	0	0	0	0	0	0
5	0	0	0	1	1	1	1	0	0	0	0	0	0	0	0
6	0	0	0	0	1	1	1	0	0	0	0	1	0	0	0
7	0	0	0	0	1	1	1	0	0	0	0	0	0	0	0
8	0	0	0	0	0	0	0	1	1	1	1	1	0	0	0
9	0	0	0	0	0	0	0	1	1	1	1	0	0	0	0
10	0	0	0	0	0	0	0	1	1	1	0	1	0	0	0
11	0	0	0	0	0	0	0	1	1	0	1	0	0	0	0
12	0	0	0	0	0	1	0	1	0	1	0	1	0	0	0
13	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1
14	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1
15	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1

Table 2. Table 2: A brief description of the tested social networks. 𝒢 𝒢 \mathcal{G} is the number of object nodes, ℐ ℐ \mathcal{I} is the number of edges, and 𝒟 ^ ^ 𝒟 \hat{\mathcal{D}} is the number of ground truth communities.

Name	$𝒢$	$ℐ$	$\widehat{𝒟}$	Description of group nature
Karate	34	78	2	Membership after the division
Football	115	615	12	Team scheduling
Dolphins	62	159	2	Group of male/female dolphins
PolBooks	105	441	3	Group of books about US politics

Table 3. Table 3: NMI-score of the considered community detection algorithms on the tested social networks. The value between (.) is the number of predicted communities.

Methods	Social networks
	Karate	Football	Dolphins	PolBooks
Louvain	$0.597$ $\left(4\right)$	$0.775$ $\left(10\right)$	$0.526$ $\left(5\right)$	$0.591$ $\left(4\right)$
CPM	$0.294$ $\left(3\right)$	$0.903$ $\left(13\right)$	$0.38$ $\left(4\right)$	$0.455$ $\left(6\right)$
COIN	0.837 $\left(2\right)$	0.978 $\left(12\right)$	1.00 $\left(2\right)$	0.885 $\left(3\right)$
GN	$0.789$ $\left(2\right)$	$0.807$ $\left(10\right)$	$0.890$ $\left(2\right)$	$0.543$ $\left(5\right)$
WalkTrap	$0.546$ $\left(5\right)$	$0.363$ $\left(10\right)$	$0.465$ $\left(8\right)$	$0.569$ $\left(4\right)$
FGM	$0.706$ (3)	$0.513$ $\left(6\right)$	$0.454$ $\left(4\right)$	$0.568$ $\left(4\right)$
InfoMAP	$0.712$ $\left(3\right)$	$0.890$ $\left(12\right)$	$0.557$ $\left(6\right)$	$0.567$ $\left(6\right)$

Equations

$$A^{\prime }=\{m\in \mathcal{M}\mid \forall g\in A,(g,m)\in \mathcal{I}\},A\subseteq \mathcal{G}$$

$$B^{\prime }=\{g\in \mathcal{G}\mid \forall m\in B,(g,m)\in \mathcal{I}\},B\subseteq \mathcal{M}$$

$$A_1\subseteq A_2⟺B_1\supseteq B_2$$

$$\sigma (c)=\frac{\mid \{e\in \mathcal{P}(A)|e^{\prime }=B\}\mid }{2^{|A|}}$$

$$\begin{array}[]{lll}\tilde{\mathbb{K}}(\mathcal{G},\mathcal{G},\mathcal{I})=\begin{cases}(g_{i},g_{j})=1&\text{If $\exists~{}(g_{i},g_{j})\in\mathcal{I},\;i\neq j$}\\ (g_{i},g_{j})=1&\text{if $\;i=j$}\\ (g_{i},g_{j})=0&\text{Otherwise}.\end{cases}\end{array}$$

$$\sigma (\tilde{c})=\frac{2^{|A|}-1}{2^{|A|}}$$

$$\forall e\in \mathcal{P}(A),e\ne \varnothing \Rightarrow e^{\prime }=A=B$$

$$\sigma (\tilde{c})=\frac{|\mathcal{P}(A)|-1}{|\mathcal{P}(A)|}=\frac{2^{|A|}-1}{2^{|A|}}$$

$$\sigma (\tilde{c})=\frac{1}{4}$$

$$G_5=(\{16,17,18,19\},\{(16,17),(16,18),(16,19)\})$$

$$NMI(\mathcal{D},\hat{\mathcal{D}})=\frac{-2{\sum }_{i=1}^{|\hat{\mathcal{D}}|}{\sum }_{j=1}^{|\mathcal{D}|}{n}_{ij}\log (\frac{n_{ij}n}{n_i{n}_j})}{{\sum }_{i=1}^{|\hat{\mathcal{D}}|}{n}_i\log (\frac{n_i}{n})+{\sum }_{j=1}^{|\mathcal{D}|}{n}_j\log (\frac{n_j}{n})}$$

Full text

K****eywords Social network analysis, community detection, Formal Concept Analysis, formal concept interestingness.

1 Introduction

Given a social network $\mathcal{U}=(\mathcal{G},\mathcal{I})$, where the node set $\mathcal{G}$ includes the objects in the social network, and the edge set $\mathcal{I}=\{(g_{i},g_{j})\mid g_{i},g_{j}\in\mathcal{G}\}$ denotes the relationship between objects. The problem is to detect all possible communities by dividing the network into groups of nodes (i.e., meaningful connected components) based on the hidden relevant relationships in $\mathcal{U}$.

The discovery of cohesive groups, cliques and communities inside a network is one of the most studied topics in social network analysis. It has attracted many researchers in sociology, biology, computer science, physics, criminology, and so on. Community detection [1, 2] aims at finding clusters as sub-graphs within a given network. A community is then a cluster where many edges connect nodes of the same group and few edges link nodes of different groups. For instance, a community in the social network LinkedIn may represent members with a similar professional profile.

Community detection algorithms can be mainly categorized into two groups [1]: (i) Agglomerative procedures, in which nodes/groups are iteratively merged if they are similar, and (ii) divisive algorithms, in which clusters are iteratively decomposed by cutting the edges between less similar vertices. A finer categorization of community detection algorithms includes the following main kinds: hierarchical clustering, modularity maximization, clique (and variants such as $n$-clique and $k$-plex) identification, block-modeling, and spectral graph partitioning [3, 4, 1].

In this paper, we leverage Formal Concept Analysis (FCA) and the stability index of identical concepts to find relevant cliques and irrelevant bridges. Formal Concept Analysis is a mathematical formalism for data analysis [5] that uses a formal context as input to construct a set of formal concepts organized in a concept lattice. It has been successfully used in several areas of computer science to discover patterns such as homogeneous groups or association rules. In [6], Freeman was the first to use FCA for community detection in one-mode data social networks. His FCA-based method starts with an adjacency matrix where objects are individuals and attributes are maximal cliques of a size at least equal to $3$, constructs the concept lattice, identifies and then eliminates special cliques and edges to finally get the communities. Indeed, this opens the door to a promising research area of using cliques to detect communities in the social network. In this context, several clique-based methods have been introduced in the literature, including clique percolation methods [7, 8, 9]. More recently, Hao et. al. [10] define a new method that identifies $k$-equiconcepts to further generate $k$-cliques in social networks. Our COIN method can be seen as akin to these clique-based methods. But a basic aspect of COIN is the use of Formal Concept Analysis theory to better understand the network topology and the exploitation of concept interestingness measure such as stability index to discover communities by identifying relevant and irrelevant parts of the social network.

The paper is organized as follows. Section 2 gives a background about social network analysis and FCA while Section 3 describes our method for community detection in one-mode data networks using FCA and interestingness measures. In Section 4 we provide an empirical of our method against existing ones. We finally conclude the paper and describe further work in Section 5.

2 Background

This section will briefly review the main concepts that support the comprehension of our COIN community detection method by using an illsutrative example, which is an excerpt of a LinkedIn connection network and contains $15$ members of the LARIM team at University of Quebec in Outaouais. As shown in Figure 1, the network is modeled as an undirected graph $\mathcal{U}=(\mathcal{G},\mathcal{I})$, where $\mathcal{G}$ is a set of $15$ nodes representing members, and $\mathcal{I}$ is a set of edges where an edge $(g_{i},g_{j})\in\mathcal{I}$ connects two members, $g_{i},g_{j}\in\mathcal{G}$, if they have a link on LinkedIn. Let us now express our basic notation.

2.1 Basic Notation and Definitions

Definition 1** (Clique).**

Let $\mathcal{U}=(\mathcal{G},\mathcal{I})$ be an undirected graph defined over the objects $\mathcal{G}$. A clique of size $k$ in $\mathcal{U}$ is a subset $Q\subset\mathcal{G}$ such that for any two nodes (i.e., objects) $g_{i}$ and $g_{j}\in Q$, there exists an edge (i.e., a binary relation) $(g_{i},g_{j})\in\mathcal{I}$.

In the sequel, we will express a clique by a set of nodes without reference to the edges. For instance, the set $Q=\{5,6,7\}$ represents a clique of size $3$.

Definition 2** (Maximal clique).**

A clique $Q\subset\mathcal{G}$ is maximal if it cannot be extended to include one more adjacent object node.

For example, $Q=\{1,2,3,4\}$ is a maximal clique of size $4$.

Definition 3** (Isolated maximal clique).**

A maximal clique $Q\subset\mathcal{G}$ is isolated if $\forall g_{i}\in Q$ and $\forall g_{j}\in\mathcal{G}-Q$ we have $(g_{i},g_{j})\notin\mathcal{I}$. That is, there is no edge that connects an object in the maximum clique $Q$ to any object outside it.

For example, $Q=\{13,14,15\}$ represents an isolated maximal clique in Figure 1.

Definition 4** (Bridge or Cut-edge).**

An edge $(g_{i},g_{j})\in\mathcal{I}$ is a bridge iff it is not contained in any cycle and its removal increases the number of connected components in the graph $\mathcal{U}$.

Definition 5** (Non-trivial Bridge).**

A Bridge $(g_{i},g_{j})\in\mathcal{I}$ is non-trivial iff its end vertices $g_{i}$ and $g_{j}$ have a degree (i.e., number of neighbors) greater than $2$.

For example, the edge $(4,5)$ is a non-trivial bridge since the end vertices $g_{4}$ and $g_{5}$ have a degree equal to $4$ and $3$, respectively.

2.2 Formal Concept Analysis

In the following we recall key notions of FCA that will be used in this paper.

Definition 6** **(Formal

context).

It is a triple $\mathbb{K}=(\mathcal{G},\mathcal{M},\mathcal{I})$, where $\mathcal{G}$ is a set of objects, $\mathcal{M}$ a set of attributes, and $\mathcal{I}$ a binary relation between $\mathcal{G}$ and $\mathcal{M}$ with $\mathcal{I}\subseteq\mathcal{G}\times\mathcal{M}$. For $g\in\mathcal{G}$ and $m\in\mathcal{M},(g,m)\in\mathcal{I}$ holds (i.e., $(g,m)=1$) iff the object $g$ has the attribute $m$, and otherwise $(g,m)\notin\mathcal{I}$ (i.e., $(g,m)=0$).

Given arbitrary subsets $A\subseteq\mathcal{G}$ and $B\subseteq\mathcal{M}$, the following derivation operators are defined:

[TABLE]

[TABLE]

where $A^{\prime{}}$ is the set of attributes common to all objects of $A$ and $B^{\prime{}}$ is the set of objects sharing all attributes from $B$. The closure operator $(.)^{\prime{}\prime{}}$ implies the double application of $(.)^{\prime{}}$, which is extensive, idempotent and monotone. The subsets $A$ and $B$ are closed when $A=A^{\prime{}\prime{}}$, and $B=B^{\prime{}\prime{}}$.

Definition 7** (Formal concept).**

The pair $c=(A,B)$ is called a formal concept of $\mathbb{K}$ with extent $A$ and intent $B$ if both $A$ and $B$ are closed and $A^{\prime{}}=B$, and $B^{\prime{}}=A$.

For a finite intent (or extent) set of $w$ elements, we use $\mathcal{P}(.)$ to denote its power set with a number of subsets equal to $n=2^{w}$, i.e., the set of all its subsets, including the empty set and the set itself.

Definition 8** (Partial order relation $\leq$).**

A concept $c_{1}=(A_{1},B_{1})$ $\leq$ $c_{2}=(A_{2},B_{2})$ if:

[TABLE]

In this case, $c_{2}$ is called a superconcept (or upper neighbor or successor) of $c_{1}$, and $c_{1}$ is called a subconcept (or lower neighbor or predecessor) of $c_{2}$. The set of all concepts of the formal context $\mathbb{K}$ is expressed by $\mathcal{C}(\mathbb{K})$ or simply $\mathcal{C}$.

Definition 9** (Concept Lattice).**

The concept lattice of a formal context $\mathbb{K}$, denoted by $\mathcal{L}(\mathbb{K})=(\mathcal{C},\leq)$, is a Hasse graphical diagram that represents all formal concepts $\mathcal{C}$ together with the partial order that holds between them. In $\mathcal{L}(\mathbb{K})$, each node represents a concept with its extent and its intent while the edges represent the partial order between concepts.

There are several methods (cf. [5, 11, 12, 13, 14]) that build the lattice, i.e., compute all the concepts together with the partial order.

One-mode data networks contain only one type of nodes and relations. Hence, we can simply adapt the formal context (in Definition 6) to define a one-mode data context as follows.

Definition 10** **(One-mode formal

context).

It is a formal context $\tilde{\mathbb{K}}=(\mathcal{G},\mathcal{G},\mathcal{I})$ in which the two sets of objects and attributes are identical, i.e., $\mathcal{G}\equiv\mathcal{M}$, and $\mathcal{I}$ is a set of relations defined on $\mathcal{G}$ with $\mathcal{I}\subseteq\mathcal{G}\times\mathcal{G}$. For $g_{i},g_{j}\in\mathcal{G}$, $(g_{i},g_{j})\in\mathcal{I}$ holds iff object $g_{i}$ is linked to $g_{j}$ or $g_{i}=g_{j}$.

2.3 Concept interestingness

Interestingness (quality) measures of a formal concept $c=(A,B)$ are commonly used to assess its relevancy. While several interestingness measures have been introduced to select relevant concepts [15, 16, 17, 18], the stability index of $c$, $\sigma(c)$ has been found to be the most prominent for selecting relevant concepts [19].

Definition 11** (Stability Index).**

Let $\mathbb{K}=(\mathcal{G},\mathcal{M},\mathcal{I})$ be a formal context and $c=(A,B)$ a formal concept of $\mathbb{K}$. The intensional stability $\sigma(c)$ can be computed as [16, 20, 15]:

[TABLE]

In Equation (2), intensional stability $\sigma(c)$ measures the strength of dependency between the intent $B$ and the objects of the extent $A$. More precisely, it expresses the probability to maintain $B$ closed when a subset of noisy objects in $A$ are deleted with equal probability. In fact, this measure quantifies the amount of noise in the extent $A$ and overfitting in the intent $B$. The numerator of $\sigma(c)$ in Eq. (2) can be computed exactly by identifying and counting the minimal generators of the concept [17, 21]. Such computation takes a time complexity of $O(L^{2})$ [17, 21], where $L$ is the size of the concept lattice, and requires the lattice construction which needs a time complexity of $O(|\mathcal{G}|^{2}\cdot|\mathcal{M}|\cdot L)$ [22, 23]. However, we have recently designed an efficient method to approximate the stability index using the low-discrepancy sampling (LDS) approach [24]. Taking a set $S$ of uniformly distributed samples from the intent powerset of a concept $c$, the LDS method needs a time complexity $\xi$ of $O(|S|)$ to estimate the stability index of $c$ with a convergence rate (i.e., sampling error) equals to O\big{(}\frac{\log|S|}{|S|}\big{)}.

3 COIN for Community Detection

At a conceptual level, our overall COIN method consists of the following key elements. First, we build the formal context and construct the concept lattice of the social network. Second, we extract from the lattice the whole set of concepts that represent all the cliques and bridges in the social graph. Third, we use a concept interestingness measure to identify noisy bridges and relevant cliques. Finally, we remove the most noisy bridges and percolate (merge) the adjacent relevant cliques to detect communities.

3.1 Building a Formal Context for a Social Network

In COIN, the first task is to build the one-mode formal context of the social network $\mathcal{U}=(\mathcal{G},\mathcal{I})$ by computing the symmetrical modified adjacency matrix [10] as follows:

[TABLE]

In Eq. (3), we assign [math] to the element of $\tilde{\mathbb{K}}$ in the row $i$ and column $j$ if the object (node) $g_{i}$ is not connected to the object $g_{j}$ in the graph $\mathcal{U}$. Otherwise, we assign $1$ to it. Note that the diagonal elements are assigned the value $1$.

For example, the constructed formal context $\tilde{\mathbb{K}}$ of our example is represented in Table 1.

We then build the concept lattice $\tilde{\mathbb{K}}$ of our example as shown in Figure 2.

3.2 Identifying Interesting cliques and Irrelevant bridges

With a little more analysis of the constructed lattice $\mathcal{L}(\tilde{\mathbb{K}})$, it is possible to identify the concepts in which the intent is equal to the extent.

Definition 12** (identical concept).**

A formal concept $c=(A,B)$, with extent $A$ and intent $B$, is called an identical concept if $A=B$, i.e., its extent and intent are identical.

We use $\tilde{\mathcal{C}}$ to denote the set of all the identical concepts.

Proposition 1**.**

Given a social graph $\mathcal{U}$ and its corresponding concept lattice $\mathcal{L}(\tilde{\mathbb{K}})$, an identical concept $c=(A,B)\in\mathcal{L}$ with $A=B$ and $|A|=k>2$, actually represents a $k$-clique $l=\{g_{i}:g_{i}\in A\}$ in $\mathcal{U}$.

Proof.

Since an identical concept is a maximal square in the formal context, it represents a unit square matrix of size $k$ - as a sub-matrix of the modified adjacency matrix - and hence a $k$-clique. Suppose now that $l=\{g_{i}\}_{i=1}^{k}$ is a $k$-clique of $\mathcal{U}$ with $k>2$. Then, from Definition 1, for any two object nodes $g_{i},g_{j}$ in $l$, there exists an edge $(g_{i},g_{j})$ in $\mathcal{U}$ that connects the two objects. Using Eq. (3), the constructed $k\times k$ modified adjacency matrix $\tilde{\mathbb{K}}(l,l,\mathcal{I}_{l})$ that expresses the clique $l$ clearly defines a matrix consisting of all 1s. Such a matrix coincides with the identical concept $\tilde{c}=(\{g_{i}\}_{i=1}^{k},\{g_{i}\}_{i=1}^{k})$ in which both extent $A$ and intent $B$ contain only the object nodes of $l$. This implies that a $k$-clique whose node set is $l=\{g_{i}:g_{i}\in A\}$ is equivalent to an identical concept $\tilde{c}=(A,B)$ such that $A=B=\{g_{i}\}_{i=1}^{k}$. ∎

For example, the identical concept $c=(\{5,6,7\},\{5,6,7\})$ of the concept lattice in Figure 2 captures the 3-clique with nodes $l=\{5,6,7\}$ of the network in Figure 1.

Therefore, from Proposition 1, we can extract the cliques of the network $\mathcal{U}$ by identifying their corresponding identical concepts in $\mathcal{L}$. For instance, the identical concepts $\tilde{\mathcal{C}}$ appear in ’yellow’ in Figure 2, and represent all the cliques and bridges as shown in Figure 3-1. Now, the role of concept interestingness comes into play. That is, we can measure how much noise exists in a clique by computing the stability index of its corresponding identical concept. The noise of a clique indicates how its objects are cohesive to each other and separable from other objects in the graph. Now, the question is how clique cohesion (looking for tightly cohesive groups) and separation (seeking for highly separated cliques) are inherited from the noise of their corresponding identical concepts. At a high level, this could be illustrated as follows. First, the stability captures the noise of an identical concept by estimating how its objects depend on the removal of each individual object. In fact, this measure of noisiness quantifies the connectivity among these objects in the corresponding clique. That is, much noise in the clique means that many objects are not cohesive and need to be removed to disconnect the clique. Second, the stability estimates the specificity of the object-object links of the identical concept with respect to the one-mode formal context. Thus, it assesses how these objects, in the corresponding clique, are influenced by the ties that hold between each individual object and other objects in the graph. That is, it approximately quantifies how these objects are strongly connected with other objects outside the clique. We call a clique relevant if it contains a very small amount of noise, i.e., its objects have a high cohesion and and a low separability.

Consequently, the stability value of an identical concept approximates the probability that its corresponding clique is a portion of a potential community. For instance, if an identical concept has the highest stability value, then the involved objects of its corresponding clique are highly cohesive and completely separable from other objects in the graph. This, in fact, renders such a clique an isolated maximum one that likely forms a stand-alone community. On the contrary, an identical concept of size $2$, which has a low stability value, could identify a very noisy 2-clique (and hence, a non-trivial bridge in the graph) that is probably not a part of any potential community.

Proposition 2**.**

Given a social graph $\mathcal{U}$ and its corresponding concept lattice $\mathcal{L}(\tilde{\mathbb{K}})$, an identical concept $\tilde{c}=(A,B)\in\mathcal{L}$ with $A=B$ and $|A|=k>2$, represents a corresponding isolated maximum k-clique $l=\{g_{i}:g_{i}\in A\}$ in $\mathcal{U}$ where $\tilde{c}$ has the highest value of the stability index:

[TABLE]

Proof.

The proposition is held once we prove that: (1) $l=\{g_{i}\}_{i=1}^{k}$ is represented by an identical concept; (2) This identical concept has the highest stability index.

(1) Suppose that $l=\{g_{i}\}_{i=1}^{k}$ is an isolated maximum clique of size $k$ in $\mathcal{U}$. Since every “isolated maximum” $k$-clique has all the properties of a $k$-clique, then, from Proposition 1, it can be easily demonstrated that $l$ has a modified adjacency matrix that defines a $k\times k$ all-ones matrix, and therefore $l$ is equivalent to an identical concept $\tilde{c}=(A,B)$ such that $A=B=\{g_{i}\}_{i=1}^{k}$.

(2) From Definitions 2 and 3 of a maximal and isolated clique, we know that there is no edge that connects any object $g_{i}$ inside $l$ to any other object $g_{a}\in\mathcal{G}\setminus l$ outside $l$. Thus, the all-ones matrix of $l$ defines a sub-matrix of the whole one-mode formal context $\tilde{\mathbb{K}}$, in which all elements in $\tilde{\mathbb{K}}$ that define the relations among the objects $\{g_{i}\}_{i=1}^{k}$ outside $l$, are zeros. From the definition of unit matrix that defines $l$, we have:

[TABLE]

That is, except the empty set, all the other elements of the powerset $\mathcal{P}(A)$ satisfy the stability condition in the numerator of Eq. (2). This implies that the numerator of Eq. (2), in the stability of $\tilde{c}$, is equal to the size of the powerset after excluding only the empty set. Thus, we have:

[TABLE]

This implies that the stability of the identical concept, $\tilde{c}=(A,B)$ with $A=B=\{g_{i}\}_{i=1}^{k}$, is equal to $\frac{2^{|A|}-1}{2^{|A|}}$, and hence increases with the size of its corresponding $k$-clique. ∎

For example, the identical concept $c=(\{13,14,15\},\{13,14,15\})$ in Figure 2 captures the isolated maximum 3-clique $l=\{13,14,15\}$ shown in Figure 1, and has its highest stability value $\sigma(\tilde{c})=\frac{2^{3}-1}{2^{3}}=0.875$.

Proposition 3**.**

Given a social graph $\mathcal{U}$ and its corresponding concept lattice $\mathcal{L}(\tilde{\mathbb{K}})$, an identical concept $c=(A,B)\in\mathcal{L}$, with $A=B=\{g_{i},g_{j}\},\;|A|=2$, represents a corresponding non-trivial bridge $(g_{i},g_{j})$ in $\mathcal{U}$, and $\tilde{c}$ has the following stability index:

[TABLE]

Proof.

The proposition is held once we prove that: (1) a bridge is represented by an identical concept with an extent and intent involving only the two objects of the bridge; (2) This identical concept has a stability value of $\frac{1}{4}$.

(1) Let $b=(g_{i},g_{j})$ be a non-trivial bridge between two components $\mathcal{T}_{i}$ and $\mathcal{T}_{j}$ of $\mathcal{U}$ such that $g_{i}\in\mathcal{T}_{i}$ and $g_{j}\in\mathcal{T}_{j}$. From Eq. (3), the $2\times 2$ modified adjacency matrix of $b$ defines a unit matrix $J_{b}$. Now, since each object of the bridge $b$ belongs to a different component, then its $J_{b}$ matrix is also a sub-matrix of the whole one-mode formal context $\tilde{\mathbb{K}}$ such that we have the following two properties:

1. (i)

$(g_{i},g_{p})=0$ $\forall g_{i}\in\mathcal{T}_{i}$ and $g_{p}\in\mathcal{T}_{j}\setminus\{g_{j}\}$ 2. (ii)

$(g_{j},g_{p})=0$ $\forall g_{j}\in\mathcal{T}_{j}$ and $g_{p}\in\mathcal{T}_{i}\setminus\{g_{i}\}$

The modified adjacency matrix $J_{b}$ of the bridge can be used to extract, from $\tilde{\mathbb{K}}$, an identical concept $\tilde{c}=(\{g_{i},g_{j}\},\{g_{i},g_{j}\})$ where both its intent and extent contain the two objects (nodes) of the bridge.

(2) $\mathcal{P}(\tilde{c})=\{\emptyset,\{g_{i}\},\{g_{j}\},\{g_{i},g_{j}\}\}$ is the powerset of the identical concept $\tilde{c}=(\{g_{i},g_{j}\},\{g_{i},g_{j}\})$. Based on the definition and properties of the modified adjacency matrix $J_{b}$ of the bridge, only one subset $\{g_{i},g_{j}\}\in\mathcal{P}(\{g_{i},g_{j}\})$ satisfies the stability condition in the numerator of Eq. (2), while the other subsets, i.e.,$\{\emptyset,\{g_{i}\},\{g_{j}\}\}$ do not. Thus, the numerator of Eq. (2) contains only one subset. This implies that the stability of the identical concept $\tilde{c}=(\{g_{i},g_{j}\},\{g_{i},g_{j}\})$ is equal to $\frac{1}{2^{|\{g_{i},g_{j}\}|}}=\frac{1}{2^{|2|}}=\frac{1}{4}$. ∎

For example, in Figure 2, the identical concept $c=(\{4,5\},\{4,5\})$ has a stability $\sigma(\tilde{c})=0.25$ and captures the non-trivial bridge $(4,5)$ in the graph of Figure 1. Object $4$ belongs to $\mathcal{T}_{i}=\{1,2,3,4\}$ while object $5$ is an element of $\mathcal{T}_{j}=\{5,6,7\}$.

3.3 COIN Algorithm For Detecting Communities

Algorithm 1 gives the pseudo-code for the COIN community detection algorithm. For clarity, Figure 3 shows the steps of COIN applied to the example in Figure 1. The algorithm takes as input the set of all identical concepts $\tilde{\mathcal{C}}$ that capture all the cliques and bridges.

The COIN algorithm then goes through two stages. At the first stage, as shown in Figure 3-2, it uses an efficient Low-discrepancy sampling (LDS) method in [24] to approximate the stability of each identical concept (line 3). Then, it distinguishes two types of identical concepts based on the estimated stability value. The first type is an identical concept that has its highest stability value. From Proposition 2, such identical concept represents an isolated community, and we therefore detect it as a community and move it into the set of final communities $\mathcal{D}$ (lines 4-7). The second type is an identical concept that contains only two objects and has a stability value of $\frac{1}{4}$ (line 8). According to Proposition 3, this identical concept represents a noisy non-trivial bridge between two potential communities. Thus, we cut this bridge by removing it from the set $\tilde{\mathcal{C}}$ (line 9). At this stage, $\tilde{\mathcal{C}}$ contains only the subset of identical concepts which have a stability higher than $\frac{1}{4}$. Such concepts capture the relevant cliques.

At the second stage, the algorithm iteratively applies a pairwise percolation of every two neighboring relevant cliques $\tilde{c}_{i},\tilde{c}_{j}\in\tilde{\mathcal{C}}$ if they share at least $a_{ij}-1$ common objects (lines 12-19), where $a_{ij}$ is the smallest number of objects in the extents of $\tilde{c}_{i}$ and $\tilde{c}_{j}$. Finally, the algorithm moves the components, obtained after the completion of percolation, into $\mathcal{D}$ (line 20), and it then returns the final set of detected communities as shown in Figure 3-3.

Assume now that the network contains an additional component that represents the following star graph:

[TABLE]

Then, three $2$-cliques (equivalent to identical concepts) are identified: $(16,17),(16,18)$ and $(16,19)$ with a stability equal to $0.5>0.25$. Since they do not represent noisy non-trivial bridges, they will be merged at the second step of the algorithm to appear as a (star) community.

Complexity analysis. The first-stage of COIN has a time complexity of $O(|\tilde{\mathcal{C}}|\times\xi)$, while the second-stage has $O(|\tilde{\mathcal{C}}|^{2})$ time complexity, where $\tilde{\mathcal{C}}$ is a set of the identical concepts and $\xi$ is the time needed to approximate the stability index of an identical concept. Thus, the total time complexity of COIN algorithm is $O(|\tilde{\mathcal{C}}|\times\xi+|\tilde{\mathcal{C}}|^{2})$.

4 experimental Evaluation

The main goal of our experimental evaluation is to investigate the following key questions:

• •

(Q1) Is COIN more accurate than the state-of-the-art community detection algorithms?

• •

(Q2) Is COIN scalable compared other prominent algorithms for detecting communities in social networks?

4.1 Methodology

We started our experiments by first selecting four datasets of real-life social networks:

Zachary’s karate club (Karate) [25] which describes $34$ members of the karate club, showing $78$ pairwise connection between members who interacted outside the club. A conflict arose between the administrator (member No. 0) and the instructor (member No. 33), which led to the scission of the club into two non-overlapped communities around the two members [math] and $33$; 2. 2.

American College football (Football) [26] which is a network that represents the schedule of games between college football teams in a single season 3. 3.

Bottlenose Dolphin (Dolphins) [27] which describes a network of frequent associations between $62$ dolphins in a community living of Doubtful Sound in New Zealand 4. 4.

Books about US politics (PolBooks) [28] is a network of books about US politics published around the time of the 2004 presidential election and sold by the online bookseller Amazon.com.

Table 2 briefly summarizes these datasets which are publicly available at the following URL111 http://www-personal.umich.edu/~mejn/netdata/..

To get answers to the proposed questions, we empirically evaluate our proposed COIN algorithm by comparing its results with the following state-of-the art community detection algorithms:

Louvain [29] which is an algorithm that uses a heuristic that maximizes the modularity 2. 2.

Clique percolation method CPM [7, 8, 9] which builds up the communities by percolating adjacent k-cliques, where two $k$-cliques are considered adjacent if they share $k-1$ object nodes 3. 3.

Girvan-Newman GN (also called Edge-betweenness) [4, 26] which is a hierarchical decomposition algorithm that progressively removes edges based on the decreasing order of their betweenness scores 4. 4.

WalkTrap [30] which performs random walks to get the walks that are more likely to stay within the same communities while only a few edges are outside a given community 5. 5.

Fast greedy modularity FGM [31] which is the fastest greedy community analysis algorithm that detects the set of communities based on the optimization of the modularity score 6. 6.

InfoMAP [32, 33]: performs random walks to analyze the information flow through a network, and detects the community set that minimizes the expected description length of a random walker trajectory.

We then assess the accuracy and scalability of results by recording the average elapsed time ($\tau$), and then calculate the following Normalized mutual information (NMI) metric [34]:

[TABLE]

Where $n=|\mathcal{G}|$ is the number of object nodes. $\mathcal{N}$ is a confusion matrix where the rows correspond to ground-truth communities $\hat{\mathcal{D}}$ and the columns correspond to predicted communities $\mathcal{D}$ found by a given community detection algorithm. Each element $n_{ij}\in\mathcal{N}$ is the number of object nodes in the i-th ground-truth community that appear in the j-th predicted community. $n_{i}=\sum_{j}n_{ij}$ is the sum over row $i$ of $\mathcal{N}$, and $n_{j}=\sum_{i}n_{ij}$ is the sum over column $j$ 222NMI metric $\in[0,1]$. That is, if the predicted communities $\mathcal{D}$ are identical to the ground-truth ones $\hat{\mathcal{D}}$, then NMI metric is equal to $1$. On the contrary, if they are totally independent, then NMI metric is [math]..

Furthermore, to guarantee a fair comparison, we conducted our empirical study with the chosen algorithms under the following setting:

• •

Re-run all tested algorithms $100$ times, and report the average result

• •

Vary the parameter $k\in\{3,4,5\}$ in the k-cliques percolation process of CPM, and report the average result

• •

Use WalkTrap to serve as a good baseline for assessing the accuracy and scalability

• •

Tune the resolution parameter $\eta$ of Louvain and FGM algorithms in the range $(0,2]$, and record the best results. Traditionally, $\eta$ is assigned a value $1$ by default. But one can set $\eta$ to a value greater than $1$ to find a larger number of smaller communities, and assign it a value less than $1$ to detect a smaller number of larger communities.

All the experiments were run on an Intel(R) Core(TM) i7-2600 CPU @ 3.20GHz computer with 16GB of memory under Windows $10$. We implemented our COIN algorithm as an extension to the FCA Concepts 0.7.11 package that is implemented by Sebastian Bank.333Publicly available at: https://pypi.python.org/pypi/concepts.

4.2 Results

In terms of accuracy, the results in Table 3 illustrate that COIN is the most accurate of all the compared algorithms, achieving the best NMI score on the four tested social networks. On the Karate dataset, it produces two communities that are approximately $84\%$ correct compared to the two ground-truth ones, and in which only the object node $31$ is wrongly detected in the ‘red’ community instead of the ‘yellow’ one as shown in Figure 5. For the Dolphins dataset, COIN detects the two communities that are identical to the ground truth ones as shown in Figure 6. On the Football dataset, COIN predicts the $12$ communities that are approximately $98\%$ correct, where only two object nodes (‘nebraska’) and (‘memphis’) are wrongly detected in ‘lime’ and ‘magenta’ communities instead of ‘green’ and ‘orange’ true communities respectively (see Figure 7). It also obtains the three communities of Polbooks dataset with $89\%$ accuracy, where as shown in Figure 8, the object nodes ‘allies’, ‘the bushes’ and ‘bush at war’ predicted to be in ‘blue’ and ‘red’ communities instead of the ‘yellow’ true one.

InfoMAP and CPM come close behind COIN on Fooball, but considerably further behind on both Dolphins and Polbooks. On both Karate and PolBooks, WalkTrap was marginally less accurate than Louvain, but more accurate than CPM. The latter outperforms both FGM and Louvain on the Football dataset. Remarkably, CPM has poor results on both the Karate and Dolphins datasets, and both InfoMAP and Louvain clearly outperform FGM on the Football dataset.

In terms of computational time, the results in Figure 4 show that COIN is relatively faster than FGM, which is the quickest community detection algorithm among all other tested ones: COIN needs only an average time of $0.186$, $2.46$, $0.61$ and $2.1$ seconds to detect communities on the four datasets respectively, while FGM takes $0.21$, $0.2.92$, $0.72$, and $2.34$ seconds. InfoMAP comes close behind FGM on all datasets, and both of them are considerably fast compared to the baseline WalkTrap on all underlying social networks. Aside from FGM algorithm, COIN clearly prevails over both CPM, Louvain and WalkTrap on all datasets by a significant margin, while saving time by more than twice compared to GN on Karate and Dolphin datasets. In practice, this is due to the fact that COIN mainly detects communities based on the set of identical-concepts $\tilde{\mathcal{C}}$, which is frequently small compared to the sets of objects and edges that are used by all other tested community detection algorithms.

Apart from COIN, FGM and InfoMAP, none of the tested algorithms — the CPM, WalkTrap, and Louvain — shows its clear superiority over the others. Remarkably, Louvain is very competitive with WalkTrap, and CPM slightly competes with GN algorithm on all datasets.

Overall, the results in Figures 4-8 and Table 3, clearly show that COIN can detect communities more accurately than existing prominent algorithms and even with less computational time.

5 Conclusion

We have proposed COIN, a two-stage method that exploits Formal Concept Analysis and concept stability to efficiently detect communities in one-mode social networks. All the identical concepts that capture cliques and bridges are first extracted from the concept lattice. Then, the stability index of these concepts is used to identify relevant cliques and cut irrelevant bridges between communities. Finally, cliques with particular features are merged to obtain the final communities. Our method has been tested on different networks with different shapes of components (e.g., star, fully or strongly connected components, chains) and led to accurate results. The empirical study on real-life social networks (see Section 4) shows that COIN can detect communities in a more accurate and efficient manner than other state-of-the-art methods.

In the future we plan to generalize COIN to detect overlapping communities both in one-mode, two-mode data and even multi-layer networks. We also intend to propose an online (incremental) variant of COIN algorithm to tackle dynamic social networks and capture the evolution of their communities over time. Finally, we intend to increase the accuracy of COIN by exploiting other concept interestingness measures (e.g. separation index and concept probability [19]) and intensively investigate the efficiency of COIN on big and dense social data networks.