Node Centrality Metrics for Hotspots Analysis in Telecom Big Data

TL;DR

This paper analyzes telecommunication big data to identify and evaluate hotspots using node centrality metrics, revealing consistent rankings over time and differences between cities and metrics.

Contribution

It compares five node centrality metrics for hotspot analysis in telecom data, providing insights into their stability and relative differences across two cities.

Findings

Hotspot rankings remain stable over two weeks.

PageRank shows smaller value differences than closeness centrality.

Variance of results is lower in Trento than Milan.

Abstract

In this work, we are interested in the applications of big data in the telecommunication domain, analysing two weeks of datasets provided by Telecom Italia for Milan and Trento. Our objective is to identify hotspots which are places with very high communication traffic relative to others and measure the interaction between them. We model the hotspots as nodes in a graph and then apply node centrality metrics that quantify the importance of each node. We review five node centrality metrics and show that they can be divided into two families: the first family is composed of closeness and betweenness centrality whereas the second family consists of degree, PageRank and eigenvector centrality. We then proceed with a statistical analysis in order to evaluate the consistency of the results over the two weeks. We find out that the ranking of the hotspots under the various centrality metrics remains practically the same with the time for both Milan and Trento. We further identify that the relative difference of the values of the metrics is smaller for PageRank centrality than for closeness centrality and this holds for both Milan and Trento. Finally, our analysis reveals that the variance of the results is significantly smaller for Trento than for Milan.

Equations

$$I_i\ge \frac{1}{N}\cdot \sum _{j=1}^N{I}_j+△,$$

$$△=\left(\mathrm{MaxTraffic}-\frac{1}{N}\cdot \sum _{j=1}^N{I}_j\right)\cdot P,$$

$$C(x)=\frac{1}{{\sum }_{y\in M(x)}d(x,y)}.$$

$$B(x)=\sum _{s\ne t\ne x}\frac{{\sigma }_{st}(x)}{{\sigma }_{st}}.$$

$$D(x)=\sum _{y\in N(x)}d(x,y).$$

$$\mathrm{PR}(\mathrm{x})=\frac{1-q}{n}+q\cdot \sum _{y\in N(x)}\frac{\mathrm{PR}(\mathrm{y})}{L(y)}.$$

$$\mathrm{Eig}(\mathrm{x})=\frac{1}{\lambda }\cdot \sum _{y\in N(x)}d(x,y).$$

$$(f\ast g)[n]=\sum _{m=-\infty }^{+\infty }f[m]\cdot g[m+n].$$

$$(f\ast f)[n]=\sum _{m=-\infty }^{+\infty }f[m]\cdot f[m+n].$$

Full text

I Introduction and related work

Nowadays, telecom companies use widely big data in order to mine the behaviour of their customers, improve the quality of service that they provide and reduce the customers’ churn. Towards this direction, demographic statistics, network deployments and call detail records (CDRs) are key factors that need to be carefully integrated in order to make accurate predictions. Though there are various open source data for the first two factors, researchers rarely have access to traffic demand data, since it is a sensitive information for the operators. Therefore, researchers need to rely on synthetic models, which do not always capture accurately large-scale mobile networks [1].

For example, the authors in [2] analyse an heterogeneous cellular network which consists of different types of nodes, such as macrocells and microcells. Nowadays a popular model is the one from Wyner [3], but it fails to fully capture a real heterogeneous cellular network because it is simplistic. Another approach is to use the spatial Poisson point process model (SPPP) [4], which can be derived from the premise that all base stations are uniformly distributed. However, a city can be classified in different areas, which have different population densities. These different areas can be characterised as dense urban, urban and suburban. To be able to classify the heterogeneous networks into these areas, the authors introduce SPPP for homogeneous and inhomogeneous sets. They show that the SPPP-model captures accurately both urban and suburban areas, whereas this is not the case for dense urban areas, because of a considerable population concentrated in small areas.

One of the few exceptions of real deployment data being publicly available is the dataset published by Telecom Italia in 2014 as ”the Big Data Challenge” [5]. This includes data collected from November to December 2013 for Milan and Trento. Our goal is to use these datasets in order to identify hotspots (areas of high communication strength) and analyse their interactions. Before proceeding with our analysis, we review the state-of-the-art.

The work closest to ours is [6]. The authors aim at finding out whether areas with happy people communicate more often with other areas of the same kind or not. To investigate this topic, they model Milan as a graph where the nodes are the areas and the weights of the edges demonstrate the communication strength between one area and another. Then, in order to estimate the happiness level of the areas, they analyse the geolocalized tweets of the dataset. They rate the happiness of a tweet, by rating the happiness of each word of a tweet and computing the average of all ratings (given to each word) for one tweet. Using this approach, they are able to differentiate between happy and unhappy tweets. The conclusion is that there is an homophily pattern: happy urban areas tend to interact with other happy areas more than they interact with unhappy areas. The same holds for unhappy areas. As in [6], we model the geographic area as a graph but we use network analysis in order to identify trends in the interactions between the areas. Moreover, we focus on hotspots, we analyse both Milan and Trento, and we use exclusively telecom data to measure the interactions between the areas.

Moreover, there have been published a number of machine learning approaches for the analysis of the Telecom Italia datasets. In [7], the authors use CDRs of the datasets in order to detect anomalies in the network (i.e., unexpected and irregular behaviour of the users) and predict future traffic using machine learning algorithms. In particular, they use k-means clustering for the detection of the region for the anomalies. The cluster with the least points is considered as the region of anomalies. They then post-process the dataset to remove the abnormal activities of the users creating anomaly-free data and train a neural network based on them that makes accurate predictions. Finally, the authors use the ARIMA model to predict future traffic for a user. In [8], the authors review learning techniques for online network optimisation for the challenging problem of load balancing of the network, concluding that there is no clear winner. Finally, in [9], the authors use unsupervised learning for the classification of mobile network usage profiles.

In this paper, we use tools from network science [10] to analyse two weeks of the Telecom Italia dataset for the cities of Milan and Trento aiming at identifying hotspots. Our contributions are three-fold: First, we show that the use of node centrality metrics [10] is a simple but powerful tool in order to analyse hotspots. We apply five of the most popular node centrality metrics and show that they can be classified into two distinct groups: The first group is composed of closeness and betweenness centrality which favours hotspots with low weights whereas the second group is composed of degree, PageRank and eigenvector centrality which favours hotspots with high weights. Secondly, our analysis reveals that the ranking of the hotspots remains practically the same under the various centrality metrics as we move from one week to the other for both Milan and Trento. Moreover, the relative difference of the values of the metrics is smaller for PageRank centrality than for closeness centrality and this holds for both Milan and Trento. Finally, we find out that the variance of the results is significantly smaller for Trento than for Milan.

II Analysis

In this section, we first briefly describe the datasets that we use from [5]. The first dataset is called ”telecommunication activity” and is composed of: square id which is the identification number of a given square of Milan/Trento grid, the approximative time of the event, incoming/outgoing amount of connections for SMS, incoming/outgoing amount of connections for call, internet traffic, and country code. This dataset is available for both Milan and Trento and will be used for the identification of the hotspots.

The second dataset is called ”Milan/Trento to Milan/Trento calls” and is composed of the following fields:

• •

Square id1: identification number of the square of Milan/Trento grid that represents the origin of the interaction.

• •

Square id2: identification number of the square of Milan or Trento grid that represents the destination of the interaction.

• •

The approximative time of the event.

• •

Directional interaction strength: value representing the directional interaction strength between Square id1 and Square id2.

This dataset will be used for the application of node centrality metrics in order to measure the communication strength since it gives information about the source and the destination of the communication whereas the first dataset only considers one area.

The last dataset that we consider is called ”Milan grid” and is composed of:

• •

square id: identification string of a given square of the Milan or Trento grid.

• •

The cell geometry expressed as geoJSON and projected in WGS84 (EPSG:4326).

This dataset is also available for both cities and will be used to visualise the hotspots.

Having completed the discussion of the datasets, we then present our approach for the identification of the hotspots. Through the ”telecommunication activity” dataset, we identify hotspots as areas with very high communication traffic. This includes the aggregated amount of connections for SMS, call and internet data. In order to quantitively determine the areas with enough high communication traffic to be considered as hotspots, we have to define a threshold. This threshold represents the minimum amount of communication traffic for an area to be considered as a hotspot and can be tuned by changing a parameter. All these requirements culminates to the following definition. We define area $i$ as a hotspot if it fulfils this inequality:

[TABLE]

where $I_{i}$ is the amount of communication for area i. The value of $\bigtriangleup$ can be calculated as follows:

[TABLE]

where MaxTraffic is the maximum amount of communication in all areas, and $P$ is a parameter to determine the cutoff threshold.

In Figs. 1a and 1b, we show an indicative visualisation of the communication traffic for the areas of Milan and Trento. We observe that the communication traffic is distributed more evenly for Trento than for Milan: the communication traffic of Milan shows a high concentration around the centre of Milan.

After extracting the hotspots, we use the following node centrality metrics [10] that quantify the importance of each hotspot: degree centrality, closeness centrality, betweenness centrality, PageRank centrality, and eigenvector centrality.

Next, we discuss the main idea of these metrics. Let $M(x)$ be the set of all nodes except node $x$, and $d(\cdot,\cdot)$ be the function for the distance between two connected nodes in a graph. For closeness centrality, we always consider the shortest distance between two nodes. Then, the closeness centrality for a node $x$ is defined as follows:

[TABLE]

Let $\sigma_{st}(x)$ be the number of the shortest paths from node $s$ to $t$ which goes through node $x$, and $\sigma_{st}$ be the number of the shortest paths from $s$ to $t$. Then, the betweenness centrality for a node $x$ is defined as follows:

[TABLE]

Let N($\cdot$) be the set of neighbours for a node. The degree centrality for a node $x$ is defined as follows:

[TABLE]

Let $\mathrm{PR}(\cdot)$ be the PageRank score for a node, $q$ be the damping factor, $n$ be the number of nodes, and $L(\cdot)$ be the number of neighbours of a node. For the PageRank centrality, we always consider the direct distance between two neighbours without going through intermediate nodes. Then, the PageRank centrality for a node $x$ is defined as follows:

[TABLE]

Finally, let $\lambda$ be the biggest eigenvalue of the adjacency matrix of the corresponding graph. Then, the eigenvector centrality for a node $x$ is defined as follows:

[TABLE]

In order to measure the importance of the hotspots, we calculate for each hotspot the different centrality metrics. We analyse the datasets for the same two weeks for both Milan and Trento so as to compare the results both over the same city and between cities. For these computations, we use the dataset ”Milan/Trento to Milan/Trento calls” because we need a source and a destination of these communication traffics in order to compute a graph and apply these metrics. Moreover, to compare effectively the results of two different weeks, we use the cross-correlation function to detect if there is a correlation between the results from the two time series. As we deal with a discrete number of hotspots, we use the discrete version of the cross-correlation function, which is defined as follows [11]:

[TABLE]

Finally, to measure the robustness of the results of the cross-correlation function, we compute the corresponding autocorrelation function that expresses the correlation of a time series with a delayed copy of itself [11]:

[TABLE]

III Results for Milan

In this section, we present the results for Milan. We start our big data analysis with the week from 18.11.2013 till 24.11.2013 (denoted as week 1) where we identify the top-20 hotspots that generate the highest traffic. Therefore, we use (1) and (2) and set experimentally the parameter $P=0.75$ in order to get exactly 20 hotspots. In Figs. 2a and 2b, we observe that the hotspots with ids 4459 and 6058 have the highest values for both closeness and betweenness centrality. On the other hand, we see from Figs. 2c, 2d and 2e that the following three hotspots (ids: 5059, 5159 and 5259) have the highest centrality values under degree, PageRank and eigenvector. Furthermore, based on the adjacent ids, we find out that these three hotspots are neighbours to each other and are situated in the centre of Milan (which was expected from the structure of the heatmap of Fig. 1a).

A general remark is that the node centrality metrics can be classified into two groups. The first group composed of closeness and betweenness centrality favours hotspots with low weights because the shortest path is a key component of both centrality metrics. The second group composed of degree, PageRank and eigenvector centrality favours hotspots with high weights because higher weights around a node is an indication of a high centrality value for this node. This can be also noticed from Fig. 2 because areas with high values for centrality metrics from one group have low scores for the centrality metric of the other group. For instance, the hotspot with the id 5159 has the lowest centrality score for the first group but the highest value for the second. This observation simplifies the analysis because we can consider one metric per family. From now on, we focus on closeness and PageRank centrality respectively.

In order to evaluate the robustness of the results (i.e., whether the trends for the hotspots are consistent across the dataset), we continue with the analysis of the dataset for week 2 (8.12.2013-14.12.2013), using the same 20 hotspots with week 1. By comparing the values of the centrality metrics for week 1 and week 2, we notice from Fig. 3 that the ranking of the hotspots for week 2 remain practically the same with week 1 for both metrics. The relative difference with respect to week 1 is less than $10\%$ for each hotspot for closeness centrality and less than $8\%$ for PageRank centrality (with the exception of one outlier for each metric). Finally, in Figs. 4a and 4b, we present the relative difference between cross-correlation and autocorrelation to quantify the difference between the observed level of similarity and the perfect one. We note that the difference is less than $6\%$ for closeness centrality and less than $2\%$ for PageRank. The above results indicate the consistency of the results since we find out that both the ranking of the hotspots and the relative difference of metrics per hotspot do not vary significantly.

IV Results for Trento

We then proceed with the analysis for Trento. We identify again the top-20 hotspots for the same two weeks that we used for Milan. In Figs. 5a and 5b, we present the results for week 1. We note that the hotspots with ids 2738 and 5202 have the highest closeness centrality values whereas the hotspots with ids 5200 and 5201 have the highest PageRank centrality values. We observe again that hotspots with high closeness centrality have low PageRank centrality and vice versa.

In Figs. 5c and 5d, we present the centrality metrics for week 2. The ranking of the hotspots remains consistent: the most important hotspots for week 2 are similar to the ones for week 1. When it comes to the relative difference of the centrality metrics with respect to week 1, we notice that, with the exception of one outlier, the relative difference is less than $8\%$ for the closeness centrality. For PageRank, the difference is always less than $2\%$. Finally, from Figs. 5e and 5f, we notice that the difference between cross-correlation and autocorrelation is always less than $4\%$ for closeness centrality and under $1\%$ for PageRank centrality.

V Conclusions

In this work, we used tools from network science to analyse two weeks of telecom big data for the cities of Milan and Trento with the view to identifying hotspots. A general conclusion from our study is that the use of node centrality metrics is a simple but powerful tool in order to analyse hotspots. Another key conclusion is that the node centrality metrics can be classified into two distinct groups: the first group is composed of closeness and betweenness centrality which favours hotspots with low weights whereas the second group is composed of degree, PageRank and eigenvector centrality which favours hotspots with high weights. Our big data analysis has shown that the ranking of the hotspots remains practically the same under the various centrality metrics as we move from one week to the other for both Milan and Trento. Moreover, we found out that the relative difference of the values of the metrics is smaller for PageRank centrality than for closeness centrality and this holds for both Milan and Trento. Finally, we found out that the variance of the results is significantly smaller for Trento than for Milan.

As a future work, we are interested in analysing the whole dataset and examine whether the conclusions from the two weeks can be generalised. Towards this direction, a natural extension is the combination of our approach with machine learning methods for traffic forecasting. Another direction is to analyse the dataset of geolocalized tweets as in [6] and compare the hotspots. Finally, it is interesting to apply the node centrality metrics to different cities with similar geographic and demographic features with Milan and Trento and evaluate the impact of these factors.