Fault Matters: Sensor Data Fusion for Detection of Faults using Dempster-Shafer Theory of Evidence in IoT-Based Applications

TL;DR

This paper introduces a sensor fault detection method for IoT applications using Dempster-Shafer Theory of Evidence, achieving high accuracy in identifying faulty sensors from benchmark and lab data.

Contribution

It applies Dempster-Shafer Theory to sensor fault detection in IoT, demonstrating superior accuracy over existing machine learning methods.

Findings

99.8% accuracy on benchmark data

99.9% accuracy on laboratory data

Effective in detecting various sensor faults

Abstract

Fault detection in sensor nodes is a pertinent issue that has been an important area of research for a very long time. But it is not explored much as yet in the context of Internet of Things. Internet of Things work with a massive amount of data so the responsibility for guaranteeing the accuracy of the data also lies with it. Moreover, a lot of important and critical decisions are made based on these data, so ensuring its correctness and accuracy is also very important. Also, the detection needs to be as precise as possible to avoid negative alerts. For this purpose, this work has adopted Dempster-Shafer Theory of Evidence which is a popular learning method to collate the information from sensors to come up with a decision regarding the faulty status of a sensor node. To verify the validity of the proposed method, simulations have been performed on a benchmark data set and data collected through a test bed in a laboratory set-up. For the different types of faults, the proposed method shows very competent accuracy for both the benchmark (99.8%) and laboratory data sets (99.9%) when compared to the other state-of-the-art machine learning techniques.

Tables

Table 1. Table 1 : Results for Faulty Node Analysis. The best values are given in bold.

	Fault Rate	10%
	$\beta$	2	4	6	8	10
		79.56					DSTE
	Offset Fault	79.71					SVM
		74.2					CART
		75.76					RF
Accuracy(%)		99.56	99.27	99.23	99.23	99.23	DSTE
	Gain Fault	79.40	78.99	78.83	78.74	78.71	SVM
		97.67	97.67	97.67	97.67	97.67	CART
		99.23	98.86	98.86	98.95	99.11	RF
		99.23					DSTE
	Out-of-Bounds Fault	79.71					SVM
		97.67					CART
		99.11					RF
		99.23					DSTE
	Data-Loss Fault	81.66					SVM
		97.67					CART
		99.11					RF
		22.29					DSTE
	Offset Fault	22.55					SVM
		28.76					CART
		26.99					RF
FPR(%)		0	0	0	0	0	DSTE
	Gain Fault	22.89	23.35	23.53	23.63	23.67	SVM
		2.28	2.28	2.28	2.28	2.28	CART
		0.52	0.94	0.94	0.84	0.66	RF
		0					DSTE
	Out-of-Bounds Fault	22.55					SVM
		2.28					CART
		0.05					RF
		0					DSTE
	Data-Loss Fault	20.34					SVM
		2.28					CART
		0.76					RF
		35.08					DSTE
	Offset Fault	35.6					SVM
		30.23					CART
		31.59					RF
Precision(%)		100	100	100	100	100	DSTE
	Gain Fault	35.26	34.8	34.63	34.53	34.5	SVM
		84.51	84.51	84.51	84.51	84.51	CART
		95.98	92.97	92.97	93.67	94.97	RF
		100					DSTE
	Out-of-Bounds Fault	35.6					SVM
		84.51					CART
		94.96					RF
		100					DSTE
	Data-Loss Fault	37.99					SVM
		84.51					CART
		95.98					RF
		94.05					DSTE
	Offset Fault	97.34					SVM
		97.34					CART
		97.34					RF
Sensitivity(%)		96.08	93.58	93.27	93.27	93.27	DSTE
	Gain Fault	97.34	97.34	97.34	97.34	97.34	SVM
		97.34	97.34	97.34	97.34	97.34	CART
		97.34	97.34	97.34	97.34	97.34	RF
		93.27					DSTE
	Out-of-Bounds Fault	97.34					SVM
		97.34					CART
		97.34					RF
		93.27					DSTE
	Data-Loss Fault	97.34					SVM
		97.34					CART
		97.34					RF
		77.71					DSTE
	Offset Fault	77.45					SVM
		71.23					CART
		72.86					RF
Specificity(%)		100	100	100	100	100	DSTE
	Gain Fault	77.10	76.64	76.46	76.36	76.32	SVM
		97.71	97.71	97.71	97.71	97.71	CART
		99.33	99.48	99.05	99.05	99.16	RF
		100					DSTE
	Out-of-Bounds Fault	77.45					SVM
		97.71					CART
		99.19					RF
		100					DSTE
	Data-Loss Fault	79.65					SVM
		97.71					CART
		99.33					RF

Table 2. Table 2 : Area under the Curve (AUC) for the Labelled Dataset

Learning Method	Offset Fault	Gain Fault	Out-of-Bounds Fault	Data-Loss Fault
DSTE	0.889	0.9982	1	1
SVM	0.8556	0.8839	0.9861	0.9941
CART	0.8548	0.9861	0.9861	0.9861
RF	0.8636	0.9941	0.9949	0.9936

Table 3. Table 3 : Comparison of Accuracy for the Different Statistical Methods. The best values are given in bold.

Fault Rate	20%
$\beta$	2	4	6	8	10
	90.47					DSTE
Offset Fault	82.33					SVM
	79.12					CART
	79.12					RF
	99.67	99.65	99.65	99.65	99.65	DSTE
Gain Fault	81.97	81.60	81.50	81.47	81.45	SVM
	99.60	99.60	99.60	99.60	99.60	CART
	99.60	99.60	99.60	99.60	99.60	RF
	99.62					DSTE
Out-of-Bounds Fault	82.33					SVM
	99.60					CART
	99.60					RF
	99.62					DSTE
Data-Loss Fault	83.7					SVM
	99.60					CART
	99.60					RF
Fault Rate	30%
$\beta$	2	4	6	8	10
	98.04					DSTE
Offset Fault	84.34					SVM
	83.52					CART
	83.06					RF
	99.70	99.68	99.68	99.67	99.67	DSTE
Gain Fault	84.06	83.74	83.63	83.61	83.58	SVM
	99.30	99.30	99.30	99.30	99.30	CART
	99.23	99.40	99.23	99.11	99.11	RF
	99.69					DSTE
Out-of-Bounds Fault	84.35					SVM
	99.27					CART
	99.07					RF
	99.70					DSTE
Data-Loss Fault	85.71					SVM
	99.23					CART
	99.30					RF
Fault Rate	40%
$\beta$	2	4	6	8	10
	99.44					DSTE
Offset Fault	85.9					SVM
	82.5					CART
	83.87					RF
	99.45	99.43	99.41	99.40	99.40	DSTE
Gain Fault	85.66	85.35	85.34	85.27	85.25	SVM
	98.34	98.34	98.34	98.34	98.34	CART
	99.32	99.32	99.32	99.32	99.32	RF
	99.71					DSTE
Out-of-Bounds Fault	85.9					SVM
	98.34					CART
	99.39					RF
	99.71					DSTE
Data-Loss Fault	89.05					SVM
	98.34					CART
	99.39					RF
Fault Rate	50%
$\beta$	2	4	6	8	10
	99.62					DSTE
Offset Fault	88.85					SVM
	86.46					CART
	87.06					RF
	99.78	99.75	99.75	99.75	99.75	DSTE
Gain Fault	88.64	88.5	88.41	88.41	88.41	SVM
	99.18	99.18	99.18	99.18	99.18	CART
	99.62	99.62	99.62	99.62	99.62	RF
	99.78					DSTE
Out-of-Bounds Fault	88.84					SVM
	99.18					CART
	99.59					RF
	99.84					DSTE
Data-Loss Fault	91.7					SVM
	99.18					CART
	99.52					RF

Table 4. Table 4 : Result of Faulty Node Analysis on Experimental Data Set (Fault Rate = 40%, α = 10 𝛼 10 \alpha=10 , β = 6 𝛽 6 \beta=6 ). The best values are given in bold.

	Accuracy (%)	FPR (%)	Precision (%)	Sensitivity (%)	Specificity (%)
	99.96	0	100	100	100	DSTE
Offset Fault	62.8	0	100	13.63	100	SVM
	99.91	0	98.29	98.87	100	CART
	99.94	0	99.19	99.65	100	RF
	99.97	0	100	100	100	DSTE
Gain Fault	62.01	66.74	53.14	100	33.25	SVM
	99.91	0	99.18	98.94	99.32	CART
	99.97	0	100	100	100	RF
	99.98	0	100	100	100	DSTE
Out-of-Bounds	62.97	0	100	13.63	100	SVM
Fault	99.91	0	99.18	98.94	100	CART
	99.96	0	100	100	100	RF
	99.98	0	100	100	100	DSTE
Data Fault	99.21	0	100	98.18	100	SVM
	99.34	0	100	98.48	100	CART
	99.96	0	100	100	100	RF

Table 5. Table 5 : Performance Evaluation of DSTE in Terms of Accuracy (%) for Different Kinds of Faults (Fault Rate = 10%, α = 10 𝛼 10 \alpha=10 , β = 2 𝛽 2 \beta=2 )

Parameters	Offset Fault	Gain Fault	Out-of-Bounds	Data Fault
			Fault
T	99.25	99.23	99.44	99.19
H	88.12	99.17	99.11	99.43
L	97.21	98.95	96.56	98.82
T+H	99.13	99.56	99.21	99.35
H+L	90.2	99.18	99.11	99.53
T+L	99.65	99.37	99.46	99.24
T+H+L	99.92	99.90	99.96	99.97
T = Temperature (0C), H = Humidity (%), L = Light (Lux)

Equations

$$\Theta =\{{\theta }_1,{\theta }_2,{\theta }_3\}$$

$$2^{\Theta }=\{\varphi ,\{{\theta }_1\},\{{\theta }_2\},\{{\theta }_3\},\{{\theta }_1{\theta }_2\},\{{\theta }_2{\theta }_3\},\{{\theta }_1{\theta }_3\},\{{\theta }_1{\theta }_2{\theta }_3\}\}$$

$$m_1\oplus m_2(Z)=\frac{{\sum }_{X\cap Y=Z\ne \varphi }{m}_1(X)m_2(Y)}{1-K}$$

$$x^{\prime }=\beta x+\eta$$

$$x^{\prime }=\alpha +x+\eta$$

$$m_{\lambda }(\delta )=\frac{{\rho }_{\lambda }(\delta )}{{\sum }_{\delta }{\rho }_{\lambda }(\delta )}$$

$${\rho }_{\lambda }(\delta )=\frac{1}{\sqrt{2\pi {\sigma }_{\delta \lambda }^2}}{\exp }^{-}\frac{(x_{t_{\lambda }}-{\mu }_{\delta \lambda }{)}^2}{2{\sigma }_{\delta \lambda }^2}$$

$${\rho }_{hum}(\delta )=\frac{1}{\sqrt{2\pi {\sigma }_{\delta hum}^2}}{\exp }^{-}\frac{(x_{t_{hum}}-{\mu }_{\delta hum}{)}^2}{2{\sigma }_{\delta hum}^2}$$

$${\rho }_{temp}(\delta )=\frac{1}{\sqrt{2\pi {\sigma }_{\delta temp}^2}}{\exp }^{-}\frac{(x_{t_{temp}}-{\mu }_{\delta temp}{)}^2}{2{\sigma }_{\delta temp}^2}$$

$${\rho }_{light}(\delta )=\frac{1}{\sqrt{2\pi {\sigma }_{\delta light}^2}}{\exp }^{-}\frac{(x_{t_{light}}-{\mu }_{\delta light}{)}^2}{2{\sigma }_{\delta light}^2}$$

$$Accuracy=\frac{\mid ({\eta }_0(t)\cap \zeta )\mid \cup \mid ({\eta }_1(t)\cap \zeta )\mid }{\mid \Delta \mid }$$

$$Sensitivity=\frac{\mid {\eta }_0(t)\cap \zeta \mid }{\mid \zeta \mid }$$

$$Specificity=\frac{\mid {\eta }_1(t)\cap \zeta \mid }{\mid \zeta \mid }$$

$$FPR=\frac{\mid {\eta }_0(t)-\zeta \mid }{\mid \Delta -\zeta \mid }$$

$$Precision=\frac{\mid {\eta }_0(t)\cap \zeta \mid }{\mid {\eta }_0(t)\mid }$$

Full text

Keywords: Classification, Data Fusion, Dempster-Shafer, Fault Detection, Internet of Things

1 Introduction

Internet of Things (IoT) have extended computing capability to everyday objects which are embedded with sensors to come up with a huge repository of data which can be exchanged and consumed without any human intervention. This data plays a huge role in decision-making policies. Thus, the foremost requirement in any application of IoT would be to ensure the reliability of data. For this purpose, detection of a faulty device which produces anomalous data is very important. A data is considered to be faulty (anomalous or outlier) if it deviates significantly from the normal range of value [9]. These deviations may turn out to be catastrophic if not detected on time. Faulty sensor data is attributed to the crash of Lion Air 737 on 29th October 2018 which killed all 189 people on-board. Erroneous signals from sensor caused continuous nose-down motions resulting in the ill-fated crash.

Outlier detection is a very pertinent topic in any field which is concerned with data like fraud and intrusion detection [8], weather monitoring [16], traffic anomaly detection [20], sensor faults in heating, ventilation and air-conditioning (HVAC) systems [14] etc. In an IoT application, monitoring and collection of real-time data from various sensors are necessary for the proper functioning of the system. All the data that is collected should be meticulously processed for the smooth operation of any system. The inherent components of any sensor network are prone to failures and this may result in incorrect readings. Like any other practical system, faulty sensor data may cause instability in IoT-based system as well. Thus, a proper fault detection technique needs to be incorporated to report the abnormality in the sensor data and thus identify a compromised node. Fault detection inherently can be considered to be a classification problem. Classification is a supervised learning technique in which labels are assigned to an observation based on its feature values. In a classification problem, data is first trained with a suitable learning technique so that the classifier learns the difference between the different classes. Later, the classifier can be used to test a new observation to make a conclusion about its class. This work uses Dempster-Shafer Theory of Evidence (DSTE) [4, 15] which is a mathematical model that combines information from different sources. DSTE deals with uncertainty and thus can work with missing informations and conflicting informations. Thus, it can be considered to be a more general approach than Bayesian theory of probability and has found its applicability in varied domains [23, 19, 3, 13, 25]. Faulty node detection which comprises of different kinds of faults as given in [12] can be yet another application where DSTE can be applied.

The main contributions of this work can thus be given as follows:

• •

Applying DSTE and devising a mass assignment function to create a classification method to detect different kinds of sensor faults in the context of IoT application.

• •

Verifying the applicability of the proposed method on the benchmark labelled data set [17] and data collected from laboratory set up.

• •

The proposed method has also been compared with existing state-of-the-art classifiers to verify its feasibility.

The rest of the paper is organised as follows: Section 2 gives the relevant literature study followed by Section 3 which gives a preliminary idea about Dempster-Shafer theory of evidence. Next, in Section 4, the different types of faults pertaining to a sensor node is elaborated. In Section 5, the proposed method is discussed and in Section 6, the results are put forward based on the experimental analysis. Finally, Section 7 concludes the paper.

2 Related Studies

There are many works which have addressed the existence of faults in sensor data but research in this context is further necessary due to exponential demand of “smart” objects and relevant IoT technologies. Fault detection technique in any system can be divided into supervised, unsupervised and semi-supervised learning. In both supervised and semi-supervised learning works with labelled data to detect the fault, whereas unsupervised learning uses patterns to sort out the faults from the rest of the data. Different approaches like Hidden Markov models, distance, clustering are available for each of the learning techniques. The authors in [18] proposed a technique to detect faults in sensors using Hidden Markov models where the experiment was carried out in real scenarios. In [5], the authors used k-nearest neighbour method for distance-based outlier detection on spatio-temporal traffic flow. Cluster-based data analysis using recursive principal component analysis (R-PCA) is proposed in [22] which aggregates redundant data from sensors and helps in detecting the outliers. Outlier mining through fuzzy set theory has been explored in [10]. Lu et.al [11] used a deep structured framework for detection of outliers. Kernel principal component analysis based Mahalanobis kernel is yet another outlier detection method that has been applied in [7]. In [21], the authors used Local Outlier Factor (LOF) algorithm to segregate normal nodes from anomalous ones. Based on density-based spatial clustering of applications with noise (DBSCAN), Abid et. al [1] proposed an outlier detection technique for wireless sensor networks. An in-network knowledge discovery approach has been proposed in [6] for outlier detection in sensor nodes to differentiate between local outlier, cluster outlier and network outlier. Threshold-based fault detection and repairing scheme with Dynamic Bayesian Network (DBN) in intelligent connected vehicle (ICV) has been explored in [24]. DBN can acquire the spatial and temporal correlations of vehicle data for precise fault detection. In [26], the authors have used Support Vector Machines for identifying faults in sensor nodes using labelled data sets. Though there have been many works which have addressed fault or outlier detection in wireless sensor networks, not many have worked towards creating a framework for an IoT platform. This work intends to achieve the same by proposing a considerable framework for an IoT environment.

3 Dempster-Shafer Theory of Evidence

Dempster-Shafer Theory of Evidence (DSTE) or the theory of belief functions is used to model and validate the uncertainty present in statistical deductions. The statical inference or deduction includes all the possible states of a system or in general the hypotheses. These hypotheses are then assigned probability assignments or mass functions and they are combined to reach a final decision. DSTE helps in fusion of sensor data by applying a combination rule on the mass functions of the data source.

3.1 Frame of Discernment

It is the set of all hypotheses say, $\theta_{1},\theta_{2},\theta_{3},\dots$ as represented by $\Theta$ where the elements are mutually exclusive and the set is exhaustive. Each subset of $\Theta$ can be considered to have a probable answer to a question. As the elements in $\Theta$ are mutually exclusive and the set is exhaustive, one only correct answer is possible. The set of all subsets is given by the power set $2^{\Theta}$. If there are three hypotheses $\theta_{1},\theta_{2},\theta_{3}$, then,

[TABLE]

and,

[TABLE]

3.2 Mass Assignment Function

A Mass Assignment Function (MAF) assigns each hypotheses $\theta_{1},\theta_{2},\theta_{3},\dots,\theta_{n}$ to a mass value $m(\theta)$ in the range [0,1] with the conditions that:

The mass value of an empty set is zero that is, $m(\phi)=0$ 2. 2.

The summation of all possible hypotheses and their combinations must be equal to 1, that is, $\sum_{\theta\in 2^{\Theta}}m(\theta)=1$

3.3 Dempster’s Combination Rule

Once individual mass assignments are done with, it becomes imperative to have a cumulative value in order to reach a conclusive decision based on sensor fusion. This problem is solved by applying Dempster’s rule which provides the tool to combine mass assignments from multiple information sources. When two mass assignments are combined they can either produce a null set or they may have an intersection point. In the former case, the mass assignment is considered to have a zero value and the mass assignment of the non-empty set is boosted by the factor $K$, commonly known as the conflict factor, such that the summation of the non-empty set is equal to 1. Considering these factors, Dempster’s rule to combine the mass assignment values from various sources is given by:

[TABLE]

Here, $K=\sum_{X\cap Y=\phi}m_{1}(X)m_{2}(Y)$ and $X,Y,Z\subseteq\Theta$. $\oplus$ is considered to be the orthogonal or direct sum. So, $m_{1}\oplus m_{2}(Z)$ is the combined belief of two separate mass assignments and $\phi$ denotes the null set. The numerator in (1) encompasses all the possibilities whose intersection is $X\cap Y=Z$. To normalise this value, it is divided by $1-K$ which represents all the combined values that produces a null set. Dempster’s rule of combination is iteratively applied on all the information sources to produce the final result.

4 Types of Faults in IoT-Based Applications

Faults can occur in any application where sensor plays a major role in data collection. The collected data is usually represented in the form of a time series $d(n,t,f(t))$ where $f(t)$ is the value of the node $n$ at time $t$. $f(t)$ can be given by: $f(t)=\alpha+\beta x+\eta$, where $\alpha$ is called the offset, $\beta$ is the gain factor, $x$ is the non-faulty data and $\eta$ represents the noise in the data at time instance $t$. The various types of faults are given as below [12]:

Gain Fault: Gain faults take place if, over a period of time, rate of change of sensed data does not match with the expected data. It can be expressed as:

[TABLE]

where, $x^{\prime}$ is the faulty reading. 2. 2.

Offset Fault: Offset faults can be attributed to the addition of an additive constant to the sensed data. It can be represented as:

[TABLE] 3. 3.

Data-Loss Fault: Data loss fault occurs when a node has missing data during a particular time series. In this case, $f(t)$ has a null value. 4. 4.

Out-of-Bounds Fault: If the normal sensed value of data lies between say, $\gamma_{1}$ and $\gamma_{2}$, then out of bounds fault will occur for $x^{\prime}\in f(t)$ if $x^{\prime}<\gamma_{1}$ and $x^{\prime}>\gamma_{2}$.

To identify these faults accurately, classification methods can be one appropriate solution. Dempster-Shafer theory is introduced next for the detection of anomalous nodes in IoT environment.

5 Proposed Method for Fault Detection

The problem of fault detection in a sensor node has been considered to be a classification problem where each class is characterised by considering the different values of the data sources. For any scenario where the detection is being performed, the raw sensor data from the sensing devices are first sent to an IoT gateway where they are aggregated or fused. This fused data is then transferred to the cloud for processing the raw data and calculating the MA values which are then combined using (1) for determining the status of a node. This decision (faulty or normal) is then intimated through the Internet using a smart phone or any other relevant technology to the end-user. Based on the decision, the user will accordingly take any relevant action.

5.1 Mass Assignment Function Calculation

Before designing the mass assignment function, first a discussion about the sensory data behaviour becomes imperative. The proposed method is based on the assumption that the sensory data is normally distributed. Mostly used for modelling complex phenomenon in statistical methods, normal or Gaussian distribution is a very common continuous probability distribution. The reasons for assuming that the data used in this work is normally distributed are many folds. Firstly, as the number of observations in this work is sufficiently large, normal distribution is a reasonable assumption. Secondly, the normal curve has a ‘bell’ shape which makes it ideal for modelling any system. Thirdly, when the data is normally distributed many results can be derived analytically. Moreover, normal distribution has similar properties like central limit theorem which states that under certain conditions it is possible to approximate large number of distributions to normal distribution when the sample size is sufficiently large. Now that the important properties of normal distribution have been discussed, it is necessary to ascertain if the data set used in this work actually follows normal distribution. This can be done through a probability plot. If the data is normally distributed, then the points in the data set should lie close to a straight line indicating a normal distribution.

The visual interpretation of probability plot are depicted in Fig. 1 and Fig. 2 from which it can be easily concluded that both the labelled and the laboratory data set follow normal distribution. The points which shows a substantive departure from the straight line are actually the outliers or the faulty data in the data set. But in general, the points lie close to the straight line thus asserting the assumption of normal distribution in this work.

Now that the data distribution is established, the mass assignment function needs to be designed. One of the most important part of Dempster-Shafer theory is the proper design of mass assignment function. In this work, the mass assignment function will depend on the underlying assumption of normal distribution. For this, at first probability density function $\rho(\delta)$ for each class $\delta=\{normal,faulty\}$ for the test vector $x_{t}$ which consists of the set of some features $\lambda$ is estimated. Probability density function is used to infer how much it is more likely that a test vector $x_{t}$ will belong to one class as compared to the other class. Based on the probability density function $\rho(\delta)$, mass assignment $m_{\lambda}(\delta)$ can be given by:

[TABLE]

where, $\rho_{\lambda}(\delta)$ is the probability density function and is given as:

[TABLE]

In (5), the explanation of the notations are as follows:

1. $\bullet$

$\sigma_{\delta\lambda}$ : Standard deviation of the training set for the feature $\lambda$ in class $\delta$ 2. $\bullet$

$\mu_{\delta\lambda}$ : Expectation of the training set for the feature $\lambda$ in class $\delta$ 3. $\bullet$

$x_{t_{\lambda}}$ : Value of feature $\lambda$ of the test vector $x_{t}$

6 Experimental Analysis

In this section, the detailed explanations of the experiments are given to show the validity of the proposed method on data collected through labelled data set [17] and laboratory experiments. The experiments have been conducted using MATLAB R2015a. The results of the proposed method using Dempster-Shafer Theory of Evidence (DSTE) has been compared with Support Vector Machine (SVM), Classification and Regression Trees (CART) and Random Forests (RF). Different fault rates with different values of $\beta$ have been applied to the data sets.

6.1 Steps of experiment

To create the mass assignment function, the probability density function $\rho_{\lambda}(\delta)$ is first calculated for both normal and faulty data set for each test vector $x_{t}$; for every test vector $x_{t}$, $\rho(normal)$ and $\rho(faulty)$ are determined. So, for each class $\delta$ = {normal,faulty} and each feature $\lambda$ = {humidity, temperature} (and light for laboratory data), $\rho_{\lambda}$ for a test vector $x_{t}$ is calculated as:

[TABLE]

[TABLE]

[TABLE]

Once these values are determined, $m_{\lambda}(\delta)$ is calculated using (4). After the calculation of $m_{\lambda}(\delta)$, the mass functions are combined according to Dempster’s rule of combination as given in (1). The inference regarding the state of a node is based on this combination rule. The decision is taken in favour of that class which has the higher evidence. This can be done by fusing the data obtained from the humidity and the temperature sensors (and light sensor for laboratory data set). Their mass functions are calculated for each class by applying the combination rule; $m_{hum}\oplus m_{temp}(normal)$ and $m_{hum}\oplus m_{temp}(faulty)$. If $m_{hum}\oplus m_{temp}(normal)>m_{hum}\oplus m_{temp}(faulty)$, then the node is normal, else it is faulty. Similarly, detection of normal and faulty node is also carried out by taking the light sensor value for laboratory data set.

6.2 Description of data sets

6.2.1 Description of the labelled data set

DSTE has also been applied on a popular labelled data set to verify its applicability. The labelled data set was prepared by the researchers of University of North Carolina at Greensboro [17]. The data consists of humidity and temperature measurements collected with the help of TelosB motes measured every 5 seconds for 6 hours. The data set was prepared with single-hop and multi-hop networks. Anomalies were introduced in the data set with the help of water kettle to change the humidity and temperature values; thus creating two classes for the data sets, normal and faulty. To apply the proposed method and the compared methods, the four types of faults as described in Section 4 have been incorporated in the data set for multi-hop networks; 1 for anomalous data and 0 for normal data. A total of 18760 observations has been aggregated to carry out the experiments. Different percentage of faulty nodes (fault rates) have been considered to prepare the final data set.

6.2.2 Data collected through laboratory setup

For further verification, experimental analysis through laboratory setup have been conducted where data is recorded using temperature, humidity and light sensors by using an arduino microcontroller system. The data were recorded in the university premises in the month of June 2019, from 9:15 in the morning till 17:30 in the evening at an interval of 13 seconds. A total of 2556 observations were recorded with three parameters. As mentioned before, faults were introduced in the data set. For all the observations, temperature, humidity and light values along with the classification category (1 for faulty and 0 for normal) and the time stamps are described.

6.3 Results

Validation Set has approach been used in this work to validate the proposed method. In validation set approach, the data set is divided into two parts; one part is called the validation set or popularly known as the hold-out set or simply the test set and the other part is the training set. Out of the total observations, 70% of the data is considered as the training set and the rest is used as a test data set.

Quantitative metrics have been used in this work for the exhaustive evaluation of the Dempster-Shafer based classifier. For classification problems, let the ground truth present in the data set $\Delta$ be $\zeta$ and the inferred faulty nodes be $\eta$. Let at any time instance $t$, the inferred faulty nodes be $\eta_{0}(t)$ and the normal nodes be $\eta_{1}(t)$. From this knowledge, the following definitions can be deduced [2].

• •

Accuracy: Accuracy gives the correctly identified normal and faulty nodes in the total data set. Accuracy can be given as:

[TABLE]

• •

Sensitivity or Recall (True Positive Rate): Sensitivity or Recall is the percentage of correctly predicted faulty nodes from the total reported faulty nodes in the data set.

[TABLE]

• •

Specificity (True Negative Rate): Specificity is the percentage of correctly identified normal nodes from the set of normal nodes in the data set.

[TABLE]

• •

False Positive Rate (FPR): FPR is the percentage of the falsely reported positives from the negatives in $\zeta$.

[TABLE]

• •

Precision: Precision is the percentage of correctly predicted faulty nodes from the total set of reported faulty nodes.

[TABLE]

The range of each of the parameters lies between [0,1]; for Accuracy, Sensitivity, Specificity and Precision, a higher value indicates better results. On the contrary, for FPR, a lower value is desired.

6.3.1 Analysis of labelled data set

The results and analysis based on the different categories of faults are described next. Table 1 gives the values for the different metrics based on different kinds of faults for DSTE, SVM, CART and RF which consider the information from both the temperature and the humidity sensor.

It can be seen from Table 1 that for the different types of faults, DSTE gives the best accuracy in most of the cases, other than the offset fault which is also quite close to the best accuracy as exhibited by SVM. DSTE shows very competitive results for false positive rate wherein for gain, out-of-bounds and data faults, it has an FPR of 0 which somewhat confirms the validity of the proposed method. As compared to the other methods, the precision rate is also quite high for DSTE. However, DSTE has a comparatively lower value of sensitivity but a very high value of specificity which may confuse the readers as to the authenticity of the proposed method. Thus, there are mixed results when all the metrics are considered together. To overcome this confusion and to determine which statistical method has the best classification ability Area under an Receiver Operating Characteristic (ROC) Curve has been taken into consideration. Area under an Receiver Operating Characteristic (AUROC) (also known as AUC (Area Under the Curve) ROC (Receiver Operating Characteristics)) exhibits the performance measurement of classification problems at different threshold settings. The curve is plotted with True Positive Rate (Sensitivity) against the False Positive Rate. It has the capability to determine the class distinguishing ability of a classifier. A higher value of the area under the curve for a classifier is an indication of its superior performance. With a value near to 1, a classifier represents a good measure of separability.

Fig. 3 shows the ROC curve for the different faults for the proposed and the compared methods with Table 2 giving the AUC values. Going by the working knowledge of AUCROC, the figures are a clear indication that DSTE outperforms all the other state-of-the-art learning methods in terms of detection of faulty nodes.

Table 3 gives the values for comparison for accuracy between the state-of-the-art statistical learning techniques with different rates of incorporated faults in the network. It can be concluded from the table that the proposed method shows the best accuracy among all the compared statistical learning techniques.

6.3.2 Analysis of laboratory data set

Experiments have also been conducted on laboratory data set for different kinds of fault detection to further validate the effectiveness of the proposed method. Table 4 shows the performance comparison of the proposed method with the existing state-of-the-art classifiers. From the table it can be seen that DSTE performs competitively better than the compared algorithms with RF also showing quite good performance.

Fig. 4 shows the accuracy of the proposed method for different kinds of faults for different percentage of faulty nodes at $\beta=2$. As can be seen from the figure, DSTE shows quite high accuracy while detecting all kinds of faults.

The details of the accuracy values for each of the faults for all combination of features or parameters are depicted in Table 5.

From the table, it is quite evident that DSTE produces quite satisfactory result for all kinds of faults. The fault detection improves when the number of fused parameters increase, thereby emphasising the importance of Dempster’s Combination rule.

7 Conclusion

IoT-based devices which are equipped with the tool to detect faulty nodes are very much the need of the hour. This work has adopted Dempster-Shafer Theory of Evidence to fuse data from different sensors to conclude about the status of a sensor node. This work is driven by the fact that fusing data from different data sources is always a better strategy than to derive conclusion form a single source. In this work, a mass assignment function is put forward which uses the concept of normal data distribution. At first the data is trained with the proposed method and then it is applied on the test cases. The proposed method is verified with a benchmark data set as well as on a real-life data set taken from laboratory set-up. Different types of faults were incorporated into both the data sets to prepare the final version of the data sets for conducting the analysis. Based on the experimental analysis, it can be concluded that the proposed method has a very high accuracy (99.8% and 99.9% for labelled and laboratory data set respectively) and also shows overall superior performance when compared to other existing machine learning techniques.

As a future work, the authors would like to create a hybrid classifier which will be an amalgamation of DSTE and other learning theories to come up with a method for detection of faulty nodes in IoT-based environment.