Android Malicious Application Classification Using Clustering

TL;DR

This paper introduces a novel clustering approach combined with machine learning classifiers to enhance the detection accuracy of Android malware, achieving over 98% overall accuracy and improving true positive and negative rates.

Contribution

It proposes a scalable clustering method that improves malware detection accuracy using machine learning classifiers on Android applications.

Findings

Achieved 98.34% overall detection accuracy with random forest.

Best true positive rate with decision tree (97.59%).

Best true negative rate with support vector machine (99.96%).

Abstract

Android malware have been growing at an exponential pace and becomes a serious threat to mobile users. It appears that most of the anti-malware still relies on the signature-based detection system which is generally slow and often not able to detect advanced obfuscated malware. Hence time-to-time various authors have proposed different machine learning solutions to identify sophisticated malware. However, it appears that detection accuracy can be improved by using the clustering method. Therefore in this paper, we propose a novel scalable and effective clustering method to improve the detection accuracy of the malicious android application and obtained a better overall accuracy (98.34%) by random forest classifier compared to regular method, i.e., taking the data altogether to detect the malware. However, as far as true positive and true negative are concerned, by clustering method, true positive is best obtained by decision tree (97.59%) and true negative by support vector machine (99.96%) which is the almost same result obtained by the random forest true positive (97.30%) and true negative (99.38%) respectively. The reason that overall accuracy of random forest is high because the true positive of support vector machine and true negative of the decision tree is significantly less than the random forest.

Tables

Table 1. Table 1: Accruacy, TPR and TNR obtained without clustering the data.

	Logistic Regression	Naive Bayes	Support Vector Machines	Decision Trees	Random Forest
Accuracy	87.96	76.66	84.68	93.70	95.82
Recall/TPR	92.44	94.89	69.38	95.27	95.40
Specificity/TNR	84.16	58.49	99.96	92.65	95.73

Table 2. Table 2: Number of clusters with different clustering algorithm and its Calinski Harabaz and Silhouette Score.

	No of Clusters	Calinski Harabaz Score	Silhouette Score
k-means Clustering	2	28568.28	0.7377
	3	25493.58	0.7005
	4	22109.38	0.6167
	5	22971.34	0.6013
Agglomerative Clustering	2	24706.58	0.7256
	3	21245.43	0.7069
	4	20046.19	0.6043
	5	20939.81	0.6052
BIRCH Clustering	2	24735.95	0.7258
	3	20844.19	0.7080
	4	18494.73	0.6656
	5	21261.10	0.6056
Gaussian Mixture Model Clustering	2	8398.36	0.4716
	3	2738.43	0.1967
	4	1640.20	0.1010
	5	4813.65	0.2519
DBSCAN Clustering	eps = 5000	1380.75	0.1539
	eps = 10000	3469.39	0.5316
	eps = 15000	3743.52	0.5349
	eps = 20000	4858.33	0.6753

Equations

$$\text{Accuracy}=\frac{\text{Number of Malware and Benign correctly classified}}{\text{Total Number of files in the dataset}}$$

$$\text{True Postive Rate (TPR)}=\frac{\text{Number of Malware correctly classified}}{\text{Number of Malware in the dataset}}$$

$$\text{True Negative Rate (TNR)}=\frac{\text{Number of Benign correctly classified}}{\text{Number of Benign in the dataset}}$$

Full text

 

Keywords: Android, Classification, Clustering, Malware Detection, Static Analysis

1 Introduction

The term Malware is derived from Malicious Software, and initially, malware was developed to show one’s technical skills, but now it has become a profit-driven industry. Over the last decade, android popularity has grown immensely, and its current mobile market share is more than 75% [2]. The popularity of android Operating System (OS) is because of its open source platform and the availability of a large number of feature-rich applications for it. These applications should ideally be benign, but because of malicious intent of the adversaries, it can be made to perform some unwanted activities in the devices, e.g., stealing the personal information, sending short message service to a premium number, spying on the user, etc.

According to G DATA Security, 3809 new malware samples were detected in 2011 [1]. Since then there has been a rapid rise in the number of android malware, and in 2017 more than three million new malware samples have been identified. In this, nearly seven million applications (70% higher as compared to 2016) were removed from Google Play which was either malware or had some unacceptable content [5]. A report published by McAfee in 2018 shows that there has been an increase of 36% in AdClick Frauds, 23% spyware and 12% Banking Trojan from the previous year [7]. Another McAfee Threat Report suggests that by the end of 2017 there were more than twenty million malware samples on the android platform out of which around $\sim$2.5 millions new samples were detected in quarter-3 itself [6]. For detection of android malware, most of the anti-malware rely on the signature-based detection techniques. But generating and maintaining signature of such a huge number malware is a herculean task. Thus in the last couple of years, researchers have actively started to explore different ways to detect the malware effectively and efficiently. Therefore recently many authors [28] [26] [16] [21] [17] have proposed machine learning as a useful technique to counter the malware, and in this field, the research can be broadly divided into two parts: the feature extraction/selection and methods for the classifications.

Feature extraction is an important component of machine learning, and for the effective and efficient classification, features can be extracted without executing the application (static) or during the execution (dynamic) for the detection of malware. Although static analysis gives a better code coverage, it has other limitations. However, with the static feature, several authors have proposed various malware detection models, e.g., Au et al. [12] in PScout extracted permissions used in 4 different android OS and found that there are over 75 unique permissions out of which 22% of the non-system permissions are unnecessary. Lindorfer et al. proposed a fully automated ANDRUBIS system to analyze the android applications from which they found that malicious application on an average request for 13 permission while for good applications the number is just 4.5 [21]. Ashu et al. used a combination of static opcode frequency and dangerous permissions for malware classification [25]. Puerta et al. with the Genome dataset found that opcode frequency is better feature vector than the permissions [16].

Malware can also be analyzed while executing them in a controlled environment to find the features, e.g., network traffic, application programming interface, system calls, information flow for the classifications. In this, Enck et al. built TaintDroid to track the runtime information flow and found many instances of private data being misuse in different applications [17]. Tam et al. developed CopperDroid to model well know process-OS interaction and also inter-process and intra-process communications for effective malware detection [26]. You et al. in 2015 did a comprehensive analysis using Dalvik opcodes as features to find the potential threats [28].

Finally, the features obtained by the static or dynamic approach is an important ingredient for the classification. In this, Puerta et al. proposed a single class classifier for malware detection based on opcode occurrence and achieved an accuracy of 85% [16]. Ashu et al. used opcode occurrence on five different classifiers and obtained the highest accuracy of 79.27% with the functional tree [24]. Feizollah et al. proposed AndroDialysis, which used intent and permission separately and then combine them to make an extensive feature vector [18]. Their analysis shows that independently intent and permission achieved a detection rate of 91% and 83% respectively. However, combing both the features shows an increase in the detection rate (95.5%). Wu et al. proposed DroidMat, which collects permission, intent, API Call and applied various machine learning algorithms like k-nearest neighbor and naive bayes for malware detection, and achieved the highest accuracy of 97.87% [27]. Arp et al. used permission, intent, API call, network address in the Drebin dataset of more than 5000 malware samples and utilized support vector machine for the classification, and achieved an accuracy of 94% [11].

Recently Ashu et al. used opcode frequency as a feature and grouped them based on permission to achieve a detection accuracy of 97.15% [25]. Li et al. used permission as the feature vector and used three level pruning for identification of significant permission for effective detection of malware and benign [20]. They have used 22-most significant permissions and achieved a detection rate of 93.62%. Rana et al. use different tree based classifiers like a decision tree, random forest, gradient boosting and extremely randomized tree with n-gram approach and shown that with 3-gram and random forest classifier one can achieve an accuracy up to 97.24% [22]. Chen et al. used n-gram opcode sequence with exemplar feature selection method and random forest classifier to detect the malware only with 95.6% correctly with 4-gram approach [15].

The above-proposed methods using different machine learning seem to be not sufficient to identify sophisticated advanced malware. However, it appears that detection accuracy can be improved by using the clustering method. Therefore in this paper, we propose a novel scalable and effective clustering method to improve the detection accuracy of the android malware. Hence, the rest of the paper is organized as follows: Section 2 introduces the dataset and feature extraction. Section 3 contains the experimental analysis and results. Finally, Section 4 concludes the paper.

2 Dataset and Feature Extraction

In the area of machine learning, it is very important that how good one can make/train a model to identify the target in new/test data, and in turn, the quality of the trained model depends on the dataset and how the features are extracted/selected from it. Understanding the fact we used Drebin dataset [11] which is one of the largest benchmark malware samples, which consists of 5550 malware samples from more than 20 different families. It also includes all the malware samples from the Android Malware Genome Project [19].

For the benign file, we have collected 8500 android applications between 2016-17 from various sources (Google Play [4], Third-party app stores, alternate marketplaces, etc.). To test the downloaded files is benign or not, we verified them using VirusTotal [10] (it is a subsidy of Alphabet which is an aggregator of 40-60 antivirus including AVG, Bitdefender, F-Secure, Kaspersky Lab, McAfee, Norton, Panda Security, etc. and provide various API’s to check whether the application is malware/benign) services, and we declare an application as malicious if one or more antivirus from virustotal.com categories it as malware. Thus after verifying all the downloaded samples from VirusTotal, we were left with 5720 samples for the experimental analysis.

For the analysis, we disassemble all the sample file by APKtool [3] to extract all the opcodes for the clustering and classifications. For this purpose, we have taken the frequency of the opcode as a feature vector and generated the feature vector for the complete dataset as $11266\times 256$ matrix, where rows represent different files, and a column represents the frequency of the opcode in that particular file.

3 Experimental Analysis and Result

A schematic of the experimental analysis of our novel proposed approach/method is depicted in figure 1. We first experimented by the regular method given by the various authors [19] [11] [27] [18] [16], i.e., without clustering/grouping the data for comparison of accuracy with our novel method (first we find the optimal number of cluster and then identify the malicious android applications), and we used the scikit-learn [9] machine learning library for all the clustering and classifications. Also Numpy python library has been used to handle large multidimensional arrays and matrices [8].

3.1 Malware Detection without clustering

To understand the performance of the different type of classifiers by regular used method (i.e., taking the whole data altogether), we used Logistic Regression (LR), Naive Bayes (NB), Support Vector Machine (SVM), Decision Tree (DT), and Random Forest (RF). For the classification, we used all 256 opcodes as features and ten-fold cross-validation (it divides the data into ten equal parts out of which nine parts are used for training the model, and one part is used for testing. This exercise is completed ten times with different combinations of training and testing set and at last average result is taken into account) and the result obtained is shown in figure 3. From the analysis, we found that after three-fold cross-validation the accuracy of classifiers is more or less same. The detailed results of the experiment conducted by all the five classifiers in terms of Accuracy, Recall, and Specificity are given in the table 1, where

[TABLE]

[TABLE]

[TABLE]

[TABLE]

The analysis also shows that the tree-based RF classifier outperformed the other classifiers with the best accuracy of 95.82%.

3.2 Clustering based Malware Detection

To improve the detection accuracy of malicious applications, we first studied the five different clustering algorithms (k-means, Agglomerative, BIRCH, DBSCAN, and GMM) to find the clusters in the dataset. The analysis shows (table 3) that the k-means algorithm forms the best cluster (Calinski-Harabasz [13] and Silhouette score [23] of the k-means is best among the selected five clustering algorithm). Further, we observed that the two clusters formed by the k-means would be best for the classification of malicious applications (fig 3) and in the obtained cluster, we find that cluster-1 contains total 8760 files and is dominated by malware files whereas the cluster-2 contains total 2780 files dominated by benign files. However, for the analysis, we have balanced the dataset using SMOTE [14]. Now for the classification, we use the same set of classifiers that are used for the classification without clustering and found that the overall accuracy, TPR and TNR obtained by clustering the data is significantly more than without clustering the data and the results obtained are given in table 3.

4 Conclusion

To detect the malware, generally signature-based anti-malware are used which is not good enough to detect the advanced obfuscated malware. Hence time-to-time to detect the advanced malware different machine learning solutions are proposed by various authors. In this, we proposed a novel scalable and effective clustering based android malware detection system. The analysis shows an overall improvement of accuracy with RF (98.34%) by our proposed clustering method. The accuracy achieved by our approach outperformed the recent accuracy obtained by the authors viz. Ashu et al. (97.15%), Li et al. (93.62%) and Rana et al. (97.24%). Also, the experimental analysis shows that despite the TNR of SVM and TPR of DT are marginally better than the RF, the overall accuracy of RF is best among the tested classifiers, this is because the TPR of SVM and TNR of DT is far below the RF classifier. As the results are significant, therefore we are developing an API for the identification of the android malicious apps, which we will be free to the research community.