A Machine Learning based Robust Prediction Model for Real-life Mobile Phone Data

TL;DR

This paper introduces a robust machine learning prediction model for real-life mobile phone data that effectively identifies and removes noisy instances to enhance prediction accuracy, using a dynamic noise threshold and decision trees.

Contribution

The paper presents a novel noise filtering approach using naive Bayes and Laplace estimator tailored to individual user behavior, improving mobile data prediction models.

Findings

Improved precision, recall, and F-measure on real mobile datasets.

Effective noise removal enhances model accuracy.

Decision tree classifier performs well on cleaned data.

Abstract

Real-life mobile phone data may contain noisy instances, which is a fundamental issue for building a prediction model with many potential negative consequences. The complexity of the inferred model may increase, may arise overfitting problem, and thereby the overall prediction accuracy of the model may decrease. In this paper, we address these issues and present a robust prediction model for real-life mobile phone data of individual users, in order to improve the prediction accuracy of the model. In our robust model, we first effectively identify and eliminate the noisy instances from the training dataset by determining a dynamic noise threshold using naive Bayes classifier and laplace estimator, which may differ from user-to-user according to their unique behavioral patterns. After that, we employ the most popular rule-based machine learning classification technique, i.e., decision tree, on the noise-free quality dataset to build the prediction model. Experimental results on the real-life mobile phone datasets (e.g., phone call log) of individual mobile phone users, show the effectiveness of our robust model in terms of precision, recall and f-measure.

Figures

Tables

Table 1. Table 1 : A sample mobile phone dataset

DayName TimeSegment	User Location	Social Situation	Social Relationship	Phone Call Behavior
Fri[S1]	Office	Meeting	Friend	Reject
Fri[S1]	Office	Meeting	Colleague	Reject
Fri[S1]	Office	Meeting	Boss	Accept
Fri[S1]	Office	Meeting	Friend	Reject
Fri[S2]	Home	Dinner	Friend	Accept
Wed[S1]	Office	Seminar	Unknown	Reject
Wed[S1]	Office	Seminar	Colleague	Reject
Wed[S1]	Office	Seminar	Mother	Accept
Wed[S2]	Home	Dinner	Unknown	Accept

Table 2. Table 2 : An example of prior probabilities

Probability	Value
P(behavior = Reject)	5/9
P(behavior = Accept)	4/9

Table 3. Table 3 : An example of conditional probabilities

Probability	Value
$P\left(DayTime=Fri\left[S1\right]|behavior=Reject\right)$	3/5
$P\left(DayTime=Fri\left[S1\right]|behavior=Accept\right)$	1/4
$P\left(DayTime=Fri\left[S2\right]|behavior=Reject\right)$	0/5
$P\left(DayTime=Fri\left[S2\right]|behavior=Accept\right)$	1/4
$P\left(DayTime=Wed\left[S1\right]|behavior=Reject\right)$	2/5
$P\left(DayTime=Wed\left[S1\right]|behavior=Accept\right)$	1/4
$P\left(DayTime=Wed\left[S2\right]|behavior=Reject\right)$	0/5
$P\left(DayTime=Wed\left[S2\right]|behavior=Accept\right)$	1/4
$P\left(Location=Office|behavior=Reject\right)$	5/5
$P\left(Location=Office|behavior=Accept\right)$	2/4
$P\left(Location=Home|behavior=Reject\right)$	0/5
$P\left(Location=Home|behavior=Accept\right)$	2/4
$P\left(Situation=Meeting|behavior=Reject\right)$	3/5
$P\left(Situation=Meeting|behavior=Accept\right)$	1/4
$P\left(Situation=Seminar|behavior=Reject\right)$	2/5
$P\left(Situation=Seminar|behavior=Accept\right)$	1/4
$P\left(Situation=Dinner|behavior=Reject\right)$	0/5
$P\left(Situation=Dinner|behavior=Accept\right)$	2/4
$P\left(Relationship=Friend|behavior=Reject\right)$	2/5
$P\left(Relationship=Friend|behavior=Accept\right)$	1/4
$P\left(Relationship=Colleague|behavior=Reject\right)$	2/5
$P\left(Relationship=Colleague|behavior=Accept\right)$	0/4
$P\left(Relationship=Boss|behavior=Reject\right)$	0/5
$P\left(Relationship=Boss|behavior=Accept\right)$	1/4
$P\left(Relationship=Mother|behavior=Reject\right)$	0/5
$P\left(Relationship=Mother|behavior=Accept\right)$	1/4
$P\left(Relationship=Unknown|behavior=Reject\right)$	1/5
$P\left(Relationship=Unknown|behavior=Accept\right)$	1/4

Table 4. Table 4 : Datasets descriptions of individual mobile phone users.

Mobile Phone Dataset (individual users)	Data Size (instances)	Time Period of User Activities
DS01	9204	9 months
DS02	5121	4 months
DS03	506	2 months
DS04	8518	9 months
DS05	2982	6 months
DS06	7040	6 months
DS07	1578	3 months
DS08	8606	4 months
DS09	7752	9 months
DS10	3798	9 months

Table 5. Table 5 : Examples of various phone call activities and corresponding contextual information

Context Category	Context Examples
Temporal Context	User’s activity occuring date (YYYY-MM-DD), time (hh:mm:ss), period (e.g., 1 hour, 10:00am-12:00pm), weekday (e.g., Monday), weekend (e.g., Saturday), etc.
Spatial Context	User’s coarse level location such as at office, work, home, market, on the way, restaurant, vehicle, playground etc.
Social Context	User’s social relationship between individuals such as mother, friend, family, colleague, boss, significant one, unknown, etc.
Call Activities	Answering an incoming call, Decline an incoming call, Missed call, and making an Outgoing call

Table 6. Table 6 : An example of noisy instances identified from the training dataset for a single iteration of N-fold.

Mobile Phone Dataset (individual users)	Noisy Instances
DS01	5.23%
DS02	3.52%
DS03	2.03%
DS04	4.12%
DS05	0.24%
DS06	4.89%
DS07	3.49%
DS08	4.70%
DS09	5.35%
DS10	4.67%

Equations

$$P(C_i|X)=\frac{P(X|C_i)P(C_i)}{P(X)}$$

$$P(X|C_i)=\prod _{k=1}^{n}P(x_k|C_i)$$

$$P(X|C_i)=P(x_1|C_i)\times P(x_2|C_i)\times ...\times P(x_n|C_i)$$

$$P(C=c_i)=\frac{n_c+k}{N+n\times k}$$

$$Precision=\frac{TP}{TP+FP}$$

$$Recall=\frac{TP}{TP+FN}$$

$$Fmeasure=2\ast \frac{Precision\ast Recall}{Precision+Recall}$$

Full text

††journal: Journal: Internet of Things: Engineering Cyber-Physical Human Systems (Elsevier),

1 Introduction

Nowadays, mobile phone is considered as an essential device of our daily life, as people around the world communicate with each other via mobile phones. According to ITU (International Telecommunication Union), nearly 96.8% of the world population has been covered by the cellular network coverage, and this coverage even increases up to 100% of the population in many countries, particularly, in the developed countries like USA, Australia, Canada, UK etc. in the world [48]. In another statistics, Sarker et al. [33] have shown that users’ interest on “Mobile Phones” is more and more than other platforms like “Desktop Computer” or “Tablet Computer” over time. People use mobile phones for various purposes such as voice communication, Internet browsing, apps using, e-mailing, SNS, instant messaging etc. [26]. The smart mobile phones have the ability to log such activities of individual mobile phone users and corresponding contextual information from different sources, such as phone logs [38], electronic calendars [40], or sensors. In this paper, we mainly focus on individuals’ diverse phone call activities in order to build our robust prediction model utilizing their real-life mobile phone data, e.g., phone call log.

Building an effective prediction model utilizing individual’s phone log data is important, as this model can be used to develop various context-aware personalized mobile applications, such as intelligent interruption management system, smart notification management system, intelligent mobile recommender system etc, in order to assist them intelligently in their daily activities. However, the real-life mobile phone data may contain noisy instances that may affect on the inferred prediction model, and thereby decrease the effectiveness in terms of prediction accuracy, in these systems. Thus, a robust prediction model is a key requirement to build such intelligent systems, which is able to handle noisy instances in mobile phone data and can perform effectively for providing personalized services to assist individuals’ in their various day-to-day situations in their real world life.

For the purpose of building a data-driven robust prediction model to predict individual’s phone call behavior, we use phone log data that consists of individuals’ diverse phone call activities, e.g., Accept, Reject, Missed, or making Outgoing call [36], and corresponding contextual information, such as temporal context, spatial context, or social context that have an influence on individuals to make a call handling decision in the real world [41]. For instance, say, on Monday, in the morning a mobile phone user typically rejects (user behavior) the incoming calls if she is in a meeting at her office; however, she accepts (user behavior) if the incoming call is from her elderly mother. Hence, “Monday morning” is an example of temporal context, “at office” represents user location which is an example of spatial context, and the interpersonal relationship “mother”, and social situation “meeting” are the social contexts relevant to that user. According to [7], “context is any information that can be used to characterize the situation of an entity, e.g., an individual mobile phone user”. Such contextual information may differ from application-to-application according to their relevancy in applications [34]. For instance, social relational context might be relevant for one application, say, call interruption management system [36], whereas, it may not be relevant for another type of application, say, mobile notification management system [21].

In the area of mining mobile phone data, both association learning [1], and classification learning [29] are the most common and popular techniques to build a rule-based user behavior model. However, association learning, e.g., Apriori [1] produces a large number of redundant rules that makes the prediction model more complex and ineffective [12] [44]. Thus, in this paper, we focus on rule-based classification technique, e.g., decision tree, that can play an important role to build an effective prediction model for individual mobile phone users utilizing their real-life mobile phone data based on multi-dimensional contexts. However, to achieve higher prediction accuracy of the inferred decision-tree based model is challenging. The reason is that classification learning technique requires a quality training data set to build an effective prediction model free from outliers or noise [6]. Typically, real-world datasets may contain noise or inconsistency instances that may cause over-fitting problem that reduce the prediction accuracy, and consequently, make the prediction model ineffective. Thus, the presence of noise in the dataset is an important issue to effectively modeling mobile user behavior [34]. According to [13], “noise is anything that obscures the relationship between the features (contexts) in an instance and it’s corresponding behavior class”. According to [13], it is evident that decision trees are badly impacted by noise. Hence, we summarize the effects of noisy instances in the real-life mobile phone data for predicting user behavior as follows:

• •

It may create unnecessary prediction rules that are not interesting to the users and make the rule-set unnecessarily larger.

• •

The complexity of the inferred decision tree based model may increase, due to the number of unnecessary training samples represented as noise.

• •

The presence of noise or inconsistency instances in the dataset may cause over-fitting problem of the decision tree-based model, and thus decrease it’s prediction accuracy.

Therefore, a noise elimination process is required before applying the decision tree classification technique to make the prediction model robust, in order to improve the prediction accuracy. According to [53], the performance of a machine learning classification technique depends on two significant factors. First one is the quality of the training data that is used to build the model, and Second one is the competence of the machine learning technique. Therefore, identification and elimination of the noisy instances from a training dataset are required to ensure the quality of the training data in order to build an effective model.

In this paper, we address the above mentioned issues and present a robust prediction model in order to improve the prediction accuracy. In our robust model, we first calculate the conditional probability for all the instances using naive Bayes classifier and laplace estimator. After that, we dynamically determine a noise threshold according to individual’s unique behavioral patterns, which may differ from user-to-user. Using this noise threshold, we identify the noisy instances that are unnecessary to build an effective model. Finally, we we employ the prominent rule-based machine learning technique, e.g., decision tree, on the noise-free quality data, to generate a set of prediction rules based on relevant multi-dimensional contexts, for the purpose of building a rule-based effective prediction model.

The contributions are summarized as follows:

• •

We determine a noise threshold dynamically by analyzing individual’s unique behavioral patterns, for the purpose of identifying inconsistency instances in the dataset.

• •

We present a machine learning based robust prediction model that effectively predicts individual’s phone call behaviors based on multi-dimensional contexts, utilizing their real-life mobile phone data.

• •

Our experiments on mobile phone datasets show that this robust model is more effective for predicting user behavior, while comparing with other base models in the area of mining mobile phone data.

This paper significantly revises and extends our earlier paper [43] in several directions: (i) defining and formulating the problem statement clearly in terms of mathematical notation; (ii) summarizing a number of base modeling approaches that utilize real-life mobile phone data; (iii) designing a decision tree based model and extracting the prediction rules based on multi-dimensional contexts; (iv) conducting additional experiments on real-life mobile phone datasets of individuals; (v) highlighting our key observations in terms of model effectiveness; and finally (vi) summarizing a number of real-world applications of the prediction model for the benefit of end mobile phone users.

The rest of the paper is organized as follows. Section 2 reviews the related work. We define and formulate the problem statement in Section 3. We give an overview about the Naive Bayes classifier and Laplace estimator that are used in our model, in Section 4 and Section 5 respectively. We present our robust prediction model in section 6. We report the experimental results in Section 7. In section 8, we discuss our key observations. We also summarize a number of real life applications of our model in Section 9 Finally, Section 10 concludes this paper and highlights the future work.

2 Related Work

Due to the popularity of smart mobile phones with advanced features and context-aware technology, analyzing real-life mobile phone data to model users’ behavioral activities, has become an active research area in the current world. A number of researchers have used mobile phone data in order to model and predict user behavior for various purposes. However, these approaches are not robust as they do not take into account the issues of noisy or inconsistence instances in the mobile phone data while building the prediction model based on contexts. For instance, Song et al. [46] present a study on users’ search behavior in order to improve searching relevance based on the log data. Rawassizadeh et al. [30] propose a scalable approach utilizing multiple sensor data, for the purpose of mining the daily behavioral patterns of the users. Mukherji et al. [22] propose an intelligent modeling approach based on multi-dimensional contexts utilizing the mobile phone data. Bayir et al. [2] propose an approach for smartphone applications in order to provide the web-based personalized service. In [24], Paireekreng et al. have proposed an approach for building personalization mobile game recommendation system. These approaches utilize mobile phone data to build their models based on multi-dimensional contexts. However, they do not take into account the robustness to build their models, in which we are interested in.

Besides these approaches, in [21], Mehrotra et al. propose a novel interruptibility management system based on mobile phone data. In their approach, they first learns users’ preferences based on multi-contexts about how the users respond with the mobile notifications utilizing mobile phone data. Xu et al. [50] have presented a prediction model for the purpose of predicting app usages behavior of mobile phone users. Zhu et al. [52] have presented an approach for the purpose of recommending mobile usages based on relevant contexts utilizing the mobile context log data. In [51], Yu et al. investigate how to exploit user context logs for the purpose of building the personalized context-aware recommendation system through topic models. In [45], Shin et al. propose an prediction model to predict mobile apps based on mobile phone data.

In addition to these approaches, a number of models based on mobile phone data have been presented for different purposes. For example, an intelligent intrusion administration framework [54], making application prefetch commonsense on cell phones [25], mining continuous co-event designs on the cell phones [47], mining versatile client propensities [20] are displayed dependent on cell phone information. Ozer et al. [23] propose a way to deal with foresee the area and time of cell phone clients by utilizing consecutive example mining systems. In [8], Do et al. present a structure for anticipating where clients will go and which application they will use in the following by misusing the rich relevant data from cell phone sensors. In [11], Farrahi et al. utilize expansive scale cell phone information for examining their day by day schedule practices. In [19], Karatzoglou et al. utilize cell phone information in their versatile application suggestion framework. Phithakkitnukoon et al. [28] utilize cell phone information in their investigation to recognize human every day action designs.

In order to predict mobile user navigation patterns, Halvey et al. [14] have done their investigation dependent on cell phone information. In their methodology, they fundamentally center around transient setting, as this is a standout amongst the most critical setting sway on client conduct. Phithakkitnukoon et al. [27] structure a versatile telephone call expectation display using cell phone information. In [18], Jang et al. have done their investigation dependent on cell phone information and demonstrated that clients application uses conduct may differ from client to client after some time. In [16], Henze et al. propose a way to deal with locate the best time to convey the portable applications dependent on cell phone information. To identify the appropriate day and age of dynamic applications, Xu et al. [49] propose a methodology by investigating the cell phone information. In [3], Bohmer et al. present a way to deal with recognize the pinnacle time of normal application utilizations as per client conduct using their cell phone information.

These approaches discussed above utilize mobile phone data to build their models for different purposes. However, they do not take into account the robustness, i.e., issues of inconsistency or noise in the datasets, in their models. Unlike these works, we present a machine learning based robust prediction model that eliminates the noisy instances from the real-life mobile phone data, in order to make the model more effective by improving the prediction accuracy.

3 Definitions and Problem Statement

This section defines the main notions concerning our robust prediction model in order to predict individuals’ diverse phone call activities utilizing their mobile phone data. In the following, the notion of mobile phone dataset having phone call activities of individuals with multi-dimensional contexts, is formally stated.

Definition 1. (Mobile phone data). Let $Con=\{con_{1},con_{2},...,con_{m}\}$ be a set of contexts and $Q=\{q_{1},q_{2},...,q_{m}\}$ the set of corresponding domains. A mobile phone dataset $DS$ is a collection of records, where - **

1. (i)

each record $r$ is a set of pairs $(con_{i},value_{i})$, where $con_{i}\in Con$, and $value_{i}\in Q$. For example, if $con_{i}$ represents the context ‘spatial’, then an example of $value_{i}$ is ‘at office’. 2. (ii)

each $con_{i}\in Con$, also called attribute (context), may occur at most once in any record in DS, and 3. (iii)

each record has a particular user activity with mobile phones (e.g., reject phone call).

The mobile phone dataset $DS$ defined above, represents the behavioral data as it contains individual’s phone call activities, e.g., rejecting an incoming phone call, in a particular context, e.g., at office.

Definition 2. (Phone Call Activity). Let, $Act=\{act_{1},act_{2},...,act_{n}\}$ be a set of activity related to the phone calls of an individual user $U$, each action $act_{i}$ represents a particular activity for that user.

In the real world, the common phone call activities of an individual mobile phone user are - (i) answering an incoming phone call, i.e., ‘Accept’, (ii) decline the incoming phone call by the user, i.e., ‘Reject’, (iii) the phone rings but the user misses the call, i.e., Missed, and (iv) making a phone call to a particular person, i.e., ‘Outgoing’ [35]. Each such phone call activity is represented as $act_{i}$ defined above.

As each activity $act_{i}$ of an individual user is associated with a particular timestamp (e.g., YYYY-MM-DD hh:mm:ss) recorded by the device, the temporal context can play a primary role to model individual’s phone call behavior. Mobile phone records such type of exact temporal information for each activity of an individual, which represents as a time-series that is continuous real-valued. A formal definition of time-series is stated in the following.

Definition 3. (Time-Series). A time-series $T_{series}$ is a sequence of data points ordered in time such that $T_{series}=(t_{1},t_{2},...,t_{m})$, where $t_{1},t_{2},...,t_{m}$ are individual observations, each of which contains real-value data and $m$ is the number of observations in a time-series [39].

To use time-series data as a temporal context in our robust prediction model, there is a need of time interval, even if only a small interval, e.g., five minutes. The reasons is that exact time is not informative for modeling and predicting user behavior [39]. Moreover, individuals’ day-wise behavior are not identical. For instance, one’s phone call behaviors on Monday might be different with her Tuesday’s behavior because of her day-to-day situations in the real world. Thus, a day-wise time segment, e.g., Friday[09:00-11:00] that represents the similar behavioral characteristics of an individual mobile phone user in that time period, could be useful. As we aim to build our robust prediction model based on multi-dimensional contexts, in the following, we define the other contexts relevant to our model.

Definition 4. (Multi-dimensional Contexts). Let $Con=\{con_{1},con_{2},...,con_{m}\}$ be a set of contexts having influence to make a phone call decision in the real world and $Q=\{q_{1},q_{2},...,q_{m}\}$ the set of corresponding domains related to phone call activity of an individual user $U$. Each context $Con_{i}$ represents a part of the multi-dimensional contexts. In addition to the temporal context discussed above, the spatial context can be considered as another dimension of contexts relevant to individual’s phone call behavior. For instance, an individual’s phone call behavior at her ‘office’ may be well different from her behavior when she is at ‘home’, which represents an example of spatial context. Some examples of these coarse level locations [32] [9] are office, home, market, store, restaurant, vehicle etc. that can be used to model individual mobile phone users’ behavior.

In addition to the above spatio-temporal contexts, social contexts or interpersonal relationship between individuals, mother, might have also an influence on individual mobile phone users to make phone call decisions [36]. For example, a user typically ‘rejects’ an incoming phone call during an event official meeting, however, she ‘answers’ if the incoming call comes from her ‘elderly mother’. Thus, the interpersonal relationship between the caller and the callee has a strong influence to handle phone call decision.

Definition 5. (Interpersonal Relationship). If $U_{1}$ is a caller and $U_{2}$ is a callee, then the social relational bonding between $U_{1}$ and $U_{2}$ represents the interpersonal relationship in their real life.

According to [36], there are several interpersonal relationships, such as mother, family, friend, colleague, boss, significant one, or unknown, can play a strong role to model user phone call behavior of individual mobile phone users in the real world.

Definition 6. (Noise). Noise is anything that obscures the relationship between the features or contexts in an instance and it’s phone call behavior class. The presence of noisy instances in the training dataset makes the machine learning based behavior prediction model ineffective in terms of prediction accuracy.

Definition 7. (Quality Mobile Phone Data). Let, $N_{total}$ is the total number of instances in the mobile phone data, $N_{noise}$ is the number of noisy instances, then $DS_{quality}=N_{total}$ - $N_{noise}$ represents the quality data that is used to build our robust prediction model for individuals.

Problem Statement. With the above definitions, the main problem we are addressing in this paper is stated as follows:

Given, a mobile phone dataset $DS$ containing real-life mobile phone data of an individual mobile phone user. Our goal is to identify and eliminate the noisy instances from $DS$ and to model individual’s behavior based on the relevant multi-dimensional contexts utilizing the quality data $DS_{quality}$, in order to predict more accurately. In this paper, we present a robust prediction model for the given dataset $DS$, using machine learning techniques, such as naive Bayes classifier, rule-based decision tree classifier, for solving this problem.

4 Naive Bayes Classifier

In our robust prediction model, we use Naive Bayes classifier (NBC) to calculate the class conditional probabilities for a given contextual information, in order to identify the noisy instances from the training dataset. NBC in one of the most popular classifiers in the area of data mining and machine learning, which is well-known as a simple probabilistic based method that is able to calculate the class membership probabilities [15]. In NBC, the effect of a contextual attribute in the dataset on a given class is also independent of those of other attributes.

Let $D$ represents a training set of data instances having the contextual attributes and corresponding behavioral class labels. Each instance in the dataset is represented by an n-dimensional context vector, $X=\{x_{1},x_{2},...,x_{n}\}$, where n represents the number of attributes of an instance and represented as $\{A_{1},A_{2},...,A_{n}\}$. Say, there are $m$ number of behavior classes in the dataset, and represented as $\{C_{1},C_{2},...,C_{m}\}$. For a test case of an instance, $X$, the NBC will predict that $X$ belongs to the class with the highest conditional probability, conditioned on $X$. That is, the naive Bayes classifier predicts that the instance $X$ belongs to the class $C_{i}$, if and only if [15] -

$P(C_{i}|X)>P(C_{j}|X)$ for $1\leq j\leq m,j\neq i$

The class $C_{i}$ for which $P(C_{i}|X)$ is maximized is called the Maximum Posteriori Hypothesis and defined as [15]:

[TABLE]

In Bayes theorem shown in Equation 1, as $P(X)$ is a constant for all classes, only $P(X|C_{i})P(C_{i})$ needs to be maximized. If the class prior probabilities are not known, then it is commonly assumed that the classes are likely equal, that is, $P(C_{1})=P(C_{2})=...=P(C_{m})$, and therefore we would maximize $P(X|C_{i})$. Otherwise, we maximize $P(X|C_{i})P(C_{i})$. The class prior probabilities are calculated by $P(C_{i})=|C_{i,D}|/|D|$, where $|C_{i,D}|$ is the number of training instances of class $C_{i}$ in $D$. To compute $P(X|C_{i})$ in a dataset with many attributes is extremely computationally expensive. Thus, the naive assumption of class-conditional independence is made in order to reduce computation in evaluating $P(X|C_{i})$. This presumes that the attributes’ values are conditionally independent of one another, given the class label of the instance, i.e., there are no dependence relationships among attributes. Thus, Equation 2 and Equation 3 are used to produce $P(X|C_{i})$ [15].

[TABLE]

[TABLE]

In Equation 2, $x_{k}$ refers to the value of attribute $A_{k}$ for instance $X$. Therefore, these probabilities $P(x_{1}|C_{i}),P(x_{2}|C_{i}),...,P(x_{n}|C_{i})$ can be easily estimated from the training instances. If the attribute value, $A_{k}$, is categorical, then $P(x_{k}|C_{i})$ is the number of instances in the class $C_{i}\in D$ with the value $x_{k}$ for $A_{k}$, divided by $|C_{i,D}|$, i.e., the number of instances belonging to the class $C_{i}\in D$.

To predict the class label of instance $X,P(X|C_{i})P(C_{i})$ is evaluated for each class $C_{i}\in D$. The naive Bayes classifier predicts that the class label of instance $X$ is the class $C_{i}$, if and only if [15] -

$P(X|C_{i})P(C_{i})>P(X|C_{j})P(C_{j})$ for $1\leq j\leq m$ and $j\neq i$

In other words, the predicted class label is the class $C_{i}$ for which $P(X|C_{i})P(C_{i})$ is the maximum. NBC has also two main advantages: (a) it requires only one scan of the training data to calculate the probability, and (b) it is easy to use to classify a class according to that probability, which makes the Naive Bayes classifier more popular in the area of classification to predict or classify the class in a given dataset. For instance, for a given particular contexts (office, meeting) in the mobile phone dataset, this classifier is able to predict the behavior (say, reject phone call) by calculating the corresponding conditional probability scanning the relevant contexts over the dataset. However, this classifier is unable to predict the actual behavior class with zero probability [15]. In our robust model, we resolve this issue by using laplace estimator that is discussed in next section.

5 Laplace Estimator

As in naive Bayes classifier discussed above, we calculate $P(X|C_{i})$ as the product of the probabilities $P(x_{1}|C_{i})\times P(x_{2}|C_{i})\times...\times P(x_{n}|C_{i})$, based on the independence assumption and class conditional probabilities, we will end up with a probability value of zero for some $P(x|C_{i})$. This may happen if the attribute value $x$ is never observed in the training data for a particular class $C_{i}$. Therefore, the generated Equation 3 becomes zeros for such attribute value regardless the values of other attributes. Thus, naive Bayes classifier is unable to predict the class of such test instances. In order to resolve such issues in our model, we use Laplace estimator [4] to scale up the values by smoothing factor. In Laplace-estimate, the class probability is defined as [4]:

[TABLE]

where $n_{c}$ is the number of instances satisfying $C=c_{i}$, $N$ is the number of training instances, $n$ is the number of classes and $k=1$.

Let’s consider a phone call behavior example, for the behavior class ‘reject’ in the training data containing 1000 instances, we have 0 instance with $relationship=unknown$, 990 instances with $relationship=friend$, and 10 instances with $relationship=mother$. The probabilities of these contexts are 0, 0.990 (from 990/1000), and 0.010 (from 10/1000), respectively. On the other hand, according to equation 4, the probabilities of these contexts would be as follows:

$\frac{1}{1003}=0.001$, $\frac{991}{1003}=0.988$, $\frac{11}{1003}=0.011$

In this way, we obtain the above non-zero probabilities, rounded up to three decimal places, respectively using Laplace estimator defined above. The “new” probability estimates are close to their “previous” counterparts, and these values can be used for further processing in our model.

6 Methodology: Our Robust Prediction Model

In this section, we present our robust prediction model for real-life mobile phone data of individual users, in order to improve the prediction accuracy of a machine learning based model.

6.1 Approach Overview

In our robust model, we first effectively identify and eliminate the noisy instances from the training dataset by determining a dynamic noise threshold that may differ from user-to-user according to their unique behavioral patterns. After that, we employ the most popular rule-based machine learning technique, i.e., decision tree, on the noise-free quality dataset to build the prediction model. Figure 1 shows an overview of our robust prediction model that utilizes real-life mobile phone data of individual users.

To achieve our goal, we first use naive Bayes classifier (NBC) discussed above as the basis for noise identification as it calculates the class conditional probability for a given contextual information. Based on such probability values for the contexts and their possible combinations in the datasets, we identify the inconsistency in the dataset. As we aim to build prediction model based on multi-dimensional contexts, such as temporal, spatial, or social context according to the given dataset, the assumption used in NBC for zero probabilities is often unrealistic for predicting individual’s phone call behavior [42], which is mentioned earlier. Thus, we further use Laplace-estimator [4] discussed above to estimate the conditional probability of contexts. After that, we calculate the “noise-threshold” using the generated probability values, in order to identify the noisy instances. As individual’s phone call behavioral patterns are not identical in the real world, this noise-threshold may differ according to individual’s unique behavioral patterns.

Once we have identified noisy instances, we eliminate those from the training dataset to get a noise-free quality data set that is used to build our robust model. After that, we employ the most popular C4.5 decision tree learning technique [29] to generate a set of rules, in order to build our robust prediction model that improves the prediction accuracy while predicting individual’s phone call behavior in multi-dimensional contexts.

6.2 Dynamic Threshold Calculation and Noise Detection

In this section, we discuss our dynamic threshold based noise detection technique using both the naive Bayes classifier (NBC) and laplace estimator, which may differ from user-to-user according to their behavioral patterns, mentioned earlier. Using NBC, we first calculate the conditional probability for each attribute by scanning the training contextual data. Table 1 shows an example of the mobile phone dataset consisting of user phone call behavior, and corresponding multi-dimensional contexts. Each instance contains four attribute or context values (e.g., time, location, situation, and social relationship between the caller and callee) and corresponding phone call behavior, e.g., Reject or Accept call activities. Based on this information, Table 2 and Table 3 report an example of the prior probabilities for each behavior class and conditional probabilities, respectively. Using these probabilities, we calculate the conditional probability for each given instance. As NBC was implemented under the independence assumption, it estimates zero probabilities if the conditional probability for a single context is zero. In such cases, we apply Laplace-estimator [4] to estimate the conditional probability of contexts.

Once we have calculated conditional probability for each instance, we differentiate between the purely classified instances and misclassified instances using these values. “Purely classified” instances are those for which the predicted class and the original class is same. If different class is found then the instances are considered as “misclassified” [10]. After that, we generate the instance groups by taking into account all the distinct probabilities as separate group values. Figure 2 shows an example of instance groups $G1,G2,G3$ for the instances ${X_{1},X_{2},...,X_{10}}$, where $G1$ consists of 5 instances with probability $p1$, $G2$ consists of 3 instances with probability $p2$, and finally $G3$ consists of 3 instances with probability $p3$. We then identify the group among the purely classified instances for which the probability is minimum. This minimum probability is considered as “noise-threshold”. Finally, the instances in misclassified list differentiated earlier, for those the probability does not satisfy the noise threshold, are identified as noise.

The process for identifying noise is set out in Algorithm 1. Input data includes training dataset: $D={X_{1},X_{2},...,X_{n}}$, which contains a set of training instances and their associated class labels and output data is the list of noisy instances. For each class, we calculate the prior probabilities $P(C_{i})$. After that for each attribute value, we calculate the class conditional probabilities $P(A_{i}|C_{i})$. For each training instance, we calculate the conditional probabilities $P(X_{i}|C_{i})$. We then check whether it is non-zero. If we get zero probabilities, we then recalculate the conditional probabilities $P(X_{i}|C_{i})$ using Laplace Estimator. Based on these probability values, we then check whether the instances are misclassified or purely classified and store all misclassified instances $misClass_{list}$ with corresponding probabilities in $misPro_{list}$. Similarly, we also store all purely classified instances $pureClass_{list}$ with corresponding probabilities in $purePro_{list}$. We then identify the minimum probability from $purePro_{list}$ as noise threshold. Finally, this algorithm returns a set of noisy instances $noise_{list}$ for a particular dataset.

Rather than arbitrarily determine the threshold, our algorithm dynamically identifies the noise threshold according to individual’s behavioral patterns and identify noisy instances based on this threshold. As individual’s phone call behavioral patterns are not identical in the real world, this noise-threshold for identifying noisy instances changes dynamically according to individual’s unique behavioral patterns.

6.3 Decision Tree based Model and Rule Generation

In this section, we discuss our decision-tree based prediction model for generating a set of rules. It is developed based on a basic C4.5 algorithm [29]. Given a training dataset, $DS=\{X_{1},X_{2},...,X_{m}\}$, where $m$ represents the sample size. Each instance is represented by an n-dimensional attribute vector, $X=\{x_{1},x_{2},...,x_{n}\}$, depicting $n$ measurements made on the instance from $n$ attributes, respectively, $\{A_{1},A_{2},...,A_{n}\}$. The training data also belong to a set of classes $C={C_{1},C_{2},...,C_{m}}$. A decision tree is a rule-based classification tree associated with $DS$, which is a structure that includes a root node, internal nodes, leaf nodes, and their associate arcs. The topmost node in the tree is called the root. Each internal node denotes a test on a context attribute, $A_{i}$, (e.g., social relationship), and each leaf node denotes the behavioral outcome class, $C_{i}$, of that test which is represented by a behavior class label (e.g., reject phone call). Each arc is associated with a particular context value, e.g., friend as a social relationship context. After building the complete decision tree, rules are extracted from the tree for the purpose of using in prediction for a particular test case.

Figure 3 shows an example of such a decision tree for multi-dimensional contexts, where locational context values are $\{home,office\}$, interpersonal social relational context values are $\{boss,friend,colleague,unknown\}$, social situation values are $\{meeting,lecture,lunch\}$, and user phone call behavior classes are $\{Accept,Reject,Missed\}$. Once the tree has been generated, rules are extracted by traversing the tree from root node to each leaf node. The followings are the examples of the produced rules $\{R_{1},R_{2},...,R_{7}\}$ generated from the tree, shown in Figure 3.

$R_{1}:{Home,Unknown\Rightarrow Missed}$

$R_{2}:{Home,Friend\Rightarrow Accept}$

$R_{3}:{Office,Meeting,Colleague\Rightarrow Reject}$

$R_{4}:{Office,Meeting,Boss\Rightarrow Accept}$

$R_{5}:{Office,Lecture\Rightarrow Reject}$

$R_{6}:{Office,Lunch,Unknown\Rightarrow Missed}$

$R_{7}:{Office,Lunch,Friend\Rightarrow Accept}$

Rule $R_{1}$ states that the user misses the incoming calls from unknown people, when she is at home. However, accepts the incoming calls at home if the calls come from her friends, which is stated in Rule $R_{2}$. Similarly, Rule $R_{3}$ states that the user rejects the incoming calls of her colleagues, when she is in a meeting at her office. However, she accepts the incoming calls from her boss, which is stated in Rule $R_{4}$. Similarly, the other rules $R_{5},R_{6},R_{7}$ state the behaviors of the user in other relevant contexts. These generated rules based on relevant multi-dimensional contexts are able to predict individual’s phone call behavior, and can be used to build a context-aware intelligent system, to assist them in their daily activities.

7 Experiments and Evaluation

In this section, we describe our experimental results for the real-life mobile phone datasets of individual users. We also present an experimental evaluation comparing our robust prediction model with the existing approaches for predicting user phone call behavior.

7.1 Real-life Mobile Phone Dataset

We have conducted experiments on ten phone log datasets collected by Massachusetts Institute of Technology (MIT) for their Reality Mining project [9]. Each dataset represents the phone call activities of an individual mobile phone user. These individuals are faculty, staff, and students. The datasets include people with different types of calling patterns and call distributions. These datasets contain multi-dimensional contexts, such as temporal, location, and social context, and corresponding phone call activities of individuals. Table 4 describes each call log dataset represented as $(DS01,DS02,...,DS10)$ for ten individual mobile phone users respectively.

7.2 Data-Preprocessing

Hence, we pre-process the raw data available in the dataset. As seen in the dataset, the mobile phone records both accepting and rejecting calls as incoming calls, we distinguish accept and reject calls by using the call duration. According to [37], if the call duration is greater than 0 then the call has been accepted; if it is equal to 0 then the call has been rejected. Overall, we take into account four types of phone call behavioral classes, such as accept/answer call, reject/decline call, missed call, and making an outgoing call, of all the individuals. In Table 5, we summarize all these activities (behavior classes) and corresponding contextual information that are taken into account in our experiments. In order to pre-process the temporal data for building this model, we use our earlier bottom-up behavior-oriented time segmentation (BOTS) technique [39] that extracts individual’s behavior oriented time segments according to the similar behavioral characteristics. For instance, Friday[09:00-11:00], Monday[12:00-13:00], Saturday[15:30-18:45] are the examples of the generated segments using this technique. We also generate data-centric social relational context using mobile phone data, where “each unique mobile phone number represents a particular one-to-one relationship” [36]. For instance, mother’s phone number (047XXXX231) represents one relation ($Rel_{1}$), while friend’s phone number (047XXXX232) represents another relation ($Rel_{2}$) etc.

7.3 Evaluation Metric

In order to measure the prediction accuracy, we compare the predicted response with the actual response (i.e., the ground truth) and compute the accuracy in terms of:

• •

Precision: ratio between the number of phone call behaviors that are correctly predicted and the total number of behaviors that are predicted (both correctly and incorrectly). If TP and FP denote true positives and false positives then the formal definition of precision is [15]:

[TABLE]

• •

Recall: ratio between the number of phone call behaviors that are correctly predicted and the total number of behaviors that are relevant. If TP and FN denote true positives and false negatives then the formal definition of recall is [15]:

[TABLE]

• •

F-measure: a measure that combines precision and recall is the harmonic mean of precision and recall. The formal definition of F-measure is [15]:

[TABLE]

7.4 Evaluation Results

To evaluate our model, we employ the most popular cross validation technique, N-fold [15], in machine learning, where we use N=10 to measure the outcome. The 10-fold cross validation breaks data into 10 sets. It trains the prediction model on 9 sets and tests it’s performance using the remaining one set. This repeats 10 times and we take a mean accuracy rate.

To show the effectiveness of our robust model, we compare the prediction results of existing approaches with our model, in terms of precision, recall and f-measure, defined earlier. In our experiment, existing approaches are those that use real-life mobile phone data for modeling without taking into account the robustness, i.e, the quality of the training data for modeling. For the purpose of effectiveness comparison with our robust model, we represent such type of existing works as ‘Base Model’, shown in Figure 4. In order to make fare comparison of both models, we use the same datasets described above, in our experiments. Figure 4 shows the overall impact of noisy instances in mobile phone data on the effectiveness of the prediction model. In Table 6, we also show the number of noisy instances identified by our approach for an iteration, which have an impact on the overall prediction accuracy. For another iteration, the number of noisy instances may change. The reason is that, we use 10-fold cross validation technique for evaluation, where 90% data is used as training data to build the model in each iteration. In another iteration, the training data is changed according to the procedure of N-fold cross validation, and consequently, the number of noisy instances may also change.

To show the overall effectiveness in terms of prediction accuracy of our robust model, Figure 4 shows the relative comparison of precision, recall and f-measure, by calculating the average results for all the datasets described earlier. If we observe Figure 4, we find that our robust model consistently outperforms base model for predicting individual’s phone call behavior in terms of precision, recall and f-measure. The main reason is that existing base models do not take into account the robustness while predicting user behavior, and the resulting accuracy is consequently low. On the other-hand, our robust model effectively handles the noisy instances by analyzing the behavioral patterns of individual mobile phone users, and thus improves the prediction accuracy in relevant contexts.

8 Discussion

Overall, our robust prediction model is completely individualized and behavior-oriented. Contrasted with the existing applicable methodologies, the user behavior prediction accuracy in terms of precision, recall and f-measure, is improved when our robust prediction model is applied, as shown in Figure 4. Although, the robust model is a two-step process, it is effective while demonstrating user behavior using real life cell phone information. The following are few key discoveries from our study.

• •

Real-world datasets may contain noisy instances. Such noisy instances may diminish the precision of expectations, increment the multifaceted nature of model induction process because of noisy training samples. To recognize such cases, determining a noise threshold according to individual’s unique behavioral patterns is the key term in our methodology. However, the threshold can differ from user-to-user, as the behavioral patterns of individual mobile phone users are not identical in the real world.

• •

Machine learning based approaches, such as using Naive Bayes classifier and Laplace Estimator can play an important role for identifying and eliminating noisy instances from the training dataset, in order to prepare the quality training data for an effective modeling. Such techniques also can help to calculate the dynamic noise-threshold by analyzing the given contextual data, in order to identify the noisy instances.

• •

We have observed a significantly lower prediction accuracy when using the base model compared to our robust model. The reason is that existing base models do not take into account the impact of noisy instances in their models. Consequently, the prediction results found using base models have low precision, recall and f-measure, shown in Figure 4.

• •

Our approach does not depend on any particular number of contexts. However, we take into account temporal, spatial, and social contexts, as these are relevant to our problem domain. For another problem domain in mobile phones, the corresponding relevant contexts can be used in our approach.

9 Real-Life Applications

As our model is able to predict future behavior of individual mobile phone users based on relevant contextual information utilizing their mobile phone data, this model can be used in various real-life applications to assist them intelligently. Hence, we summarize a number of relevant applications. These are:

• •

Intelligent Voice Communication: Managing incoming call interruptions can be an important application of this model. For this, phone call log data having different types of behaviors with incoming phone calls in different contexts, is needed to build the model. Such application will be helpful for individual mobile phone users to assist them in their daily activities, as mobile phones are considered to be ‘always on, always connected’ device in the real world, but the users are not always attentive and responsive to incoming phone calls, because of their various day-to-day situations [5]. Such call intrusions may make humiliating circumstance in an official situation, e.g., meeting, as well as influence in different exercises like inspecting patients by a specialist or driving a vehicle and so on. These call interferences may likewise diminish laborer execution, expanded blunders and worry in a workplace [26]. Therefore, in order to minimize such interruptions, a robust model utilizing individual’s phone call response behavioral data collected from related sources having relevant contexts such as temporal, spatial, social or interpersonal relationship between caller and callee, can be used to build an intelligent call interruption management system. Similarly, building robust model utilizing outgoing call related data can be used to build a smart call reminder system that can help to keenly looks through the attractive contact from the substantial contact list and reminds the user to make a phone call to a specific individual in a specific settings.

• •

Mobile Notification Management: Notifications are a core feature of mobile phones [31]. In the real world, a variety of smart mobile applications use notifications in order to inform the the users about various kinds of events, such as the arrival of a message, a new comment on their posts on social networks, like Facebook, LinkedIn, Twitter, the updating availability of an application like WhatsApp, Skype, Viber, various news or just to send them reminders or alerts [31]. However, many of them (mobile phone notifications) are neither useful nor relevant to the interests of the users. As a result, such useless notifications are considered disruptive and potentially annoying to the users [21] [17]. Individual’s behavioral rules based on user’s contextual information, might be able to manage such notifications intelligently. For examples, one individual always dismisses promotional email notifications; she accepts birthday reminder notifications of Facebook mostly at night, when she is at home; does not accept Viber or Whatsapp notifications from unknown persons at office. Therefore, building robust model utilizing individual’s notification handling behavioral data collected from related sources having relevant contexts such as temporal, spatial, notification type, can be used to build an intelligent notification management system.

• •

Mobile Recommendation System: Mobile recommender system is one of the most important applications of user behavior model, as it helps the user to find the most satisfying service by reducing search effort and information overload. For instance, a particular mobile app recommendation among a huge number of installed apps (e.g., Multimedia, Facebook, Gmail, Youtube, Skype, Game, Microsoft outlook, etc.) according to users current contextual information (temporal, spatial or others), and preference could be useful for the users. For this relevant mobile phone data having various contexts and corresponding app usages can be used for building the app recommendation model for the purpose of identifying which app is preferred by a particular user under a certain context to provide personalized context-aware recommendation of different mobile phone apps for the mobile phone users. Similarly, other recommendation services can be generated from relevant mobile phone data using our robust model in order to get better prediction accuracy in various contexts.

10 Conclusion and Future Work

In this paper, we have presented a robust prediction model for the real-life mobile phone data, in order to improve the prediction accuracy. In our robust model, we have effectively handled the noisy instances from the training dataset by determining a dynamic noise threshold using naive Bayes classifier and laplace estimator, which may differ from user-to-user according to their unique behavioral patterns. In order to build the complete model, we have also employed the most popular rule-based machine learning technique, i.e., decision tree, on the noise-free quality dataset. Experimental results on multi-contextual phone call log datasets indicate that compare to the existing approaches, our robust model improves the prediction accuracy in terms of precision, recall and f-measure.

In future work, we plan to use this model for building real-world mobile applications discussed above, to assess the effectiveness of this model in application level.

Acknowledgment

The author would like to thank Prof. Jun Han, Swinburne University of Technology, Australia, Dr. Alan Colman, Swinburne University of Technology, Australia, and Dr. Ashad Kabir, Charles Sturt University, Australia for their relevant discussions.