A novel framework of the fuzzy c-means distances problem based weighted distance

TL;DR

This paper introduces a new Canberra weighted distance metric to enhance the fuzzy c-means clustering algorithm, addressing issues with Euclidean distance in multidimensional and noisy data, and demonstrates improved performance on UCI datasets.

Contribution

The paper proposes a novel Canberra weighted distance metric to improve FCM clustering accuracy over traditional Euclidean distance.

Findings

Proposed method outperforms original FCM on UCI datasets.

Canberra weighted distance reduces clustering errors in noisy data.

Enhanced clustering results demonstrate the effectiveness of the new distance metric.

Abstract

Clustering is one of the major roles in data mining that is widely application in pattern recognition and image segmentation. Fuzzy C-means (FCM) is the most used clustering algorithm that proven efficient, fast and easy to implement, however, FCM uses the Euclidean distance that often leads to clustering errors, especially when handling multidimensional and noisy data. In the last few years, many distances metric have been proposed by researchers to improve the performance of the FCM algorithms, and the majority of researchers propose weighted distance. In this paper, we proposed Canberra Weighted Distance to improved performance of the FCM algorithm. The experimental result using the UCI data set show the proposed method is superior to the original method and other clustering methods.

Tables

Table 1. Table 1: Pseudocode of the fuzzy c-means

Algorithm 1 : Fuzzy c-means
Input: Datasets $x(i,j)$ Output: terminal fuzzy partition matrix $\mu (i,k)$ Begin
1	Input dataset $x(i,j)$
2	Initialization number of clusters c, fuzziness exponent m, termination tolerance $\epsilon$ and maximum iteration MaxIter;
3	Initialization fuzzy partition matrix $\mu (i,k)$ randomize;
4	Repeat
5		Compute the cluster centers using Eq. (4);
6		Calculate distance using Eq. (3);
7		Compute objective function using Eq. (1);
		Update fuzzy partition matrix using Eq. (2);
	Until $\left(\left|P\left(t\right)-P\left(t-1\right)\right|or\left(t>MaxIter\right)\right)$
End

Table 2. Table 2: Pseudocode of the proposed method

Algorithm 1 : Fuzzy c-means
Input: Datasets $x(i,j)$ Output: terminal fuzzy partition matrix $\mu (i,k)$ Begin
1	Input dataset $x(i,j)$
2	Initialization number of clusters c, fuzziness exponent m, termination tolerance $\epsilon$ and maximum iteration MaxIter;
3	Initialization fuzzy partition matrix $\mu (i,k)$ randomize;
4	Repeat
5		Compute the cluster centers using Eq. (4);
6		Calculate Canberra weighted distance using Eq. (13);
7		Compute objective function using Eq. (1);
		Update fuzzy partition matrix using Eq. (2);
	Until $\left(\left|P\left(t\right)-P\left(t-1\right)\right|or\left(t>MaxIter\right)\right)$
End

Table 3. Table 3: Dataset description

Datasets	Instances	Attrb.	Class	Distribution
Iris	150	4	3	50, 50, 50
Sonar	208	60	2	97, 111
WDBC	569	30	2	357, 212
WBC	699	9	2	458, 241
Wholesale Customers	582	7	2	298, 142
Abalone	4177	8	3	1528, 1307, 1342
Tae	100	5	3	22, 33, 45
Yeast	1484	8	10	244, 429, 463, 44, 35, 51, 163, 30, 20, 5
Bufa	245	6	2	145, 200, 345

Table 4. Table 4: Computer specs

Items	Specs
Processor	Intel Core i5-5200U 2.7 Ghz
Memory	6 GB
Hard Disk	1 TB
Operation System	Windows 10 Professional 64 bit
Software apps	Ms. Visual Studio 2012
	RapidMiner Studio 7.5
	XLSTAT

Table 5. Table 5: Comparison error rates and misclassification with other method

Authors	Feature weight	ER	MC
	SL, SW, PL, PW
Bezdek 1981 [10]	-	10.667	16
Wang et al. 2004 [22]	0.0001, 0.0002, 1, 0.164	5.333	8
Hung et al. 2008 [23]	0.102, 0.1022, 0.3377, 0.458	4.667	7
Zhang et al. 2014 [24]	0.0571, 0.1058, 0.4505, 0.3867	3.333	5
Xing et al. 2014 [25]	0.1194, 0.1134, 0.4346, 0.3327	3.333	5
Proposed method	0.0728, 0, 1, 0.5534	2.667	4

Table 6. Table 8: Comparison of average iteration

Datasets	Initialize
	Random	Proposed
Iris	18	10
WDBC	28	23
Total	46	33

Table 7. Table 11: Friedman’s test of error rates evaluation

Q (Observed value)	11.2903
Q (Critical value)	5.9915
DF	2
p-value (Two-tailed)	0.0035
alpha	0.05

Table 8. Table 12: P-values comparisons using nemenyi’s procedure

	FCM	IFWFCM	CWFCM
FCM	1	0.8259	0.0090
IFWFCM	0.8259	1	0.0483
CWFCM	0.0090	0.0483	1

Equations

$$P_t=\sum _{i=1}^n\sum _{k=1}^c{\mu }_{ik}^d{d}_{ij}^2$$

$${\mu }_{ik}=\frac{{\left(d_{ij}^2\right)}^{\frac{-1}{z-1}}}{{\sum }_{k=1}^c{\left(d_{ij}^2\right)}^{\frac{-1}{z-1}}}$$

$$d_{ij}^2=\sum _{ij}^m{\left|x_{ij}-v_{kj}\right|}^2$$

$$v_{kj}=\frac{{\sum }_{i=1}^n{\mu }_{ik}^m{x}_{ij}}{{\sum }_{i=1}^n{\mu }_{ik}^m}$$

$$d_{ij}=\sum _{j=1}^m\left|x_{ij}-v_{kj}\right|$$

$$d_{ij}=\sum _{j=1}^m\sqrt[p]{{\left|x_{ij}-v_{kj}\right|}^p}$$

$$d_{ij}=\sum _{j=1}^m\frac{\left|x_{ij}-vkj\right|}{\left|x_{ij}\right|+\left|v_{kj}\right|}$$

$$d_{ij}={\left(x_{ij}-v_{kj}\right)}^{T}A\left(x_{ij}-v_{kj}\right)$$

$$VMR=s^2/x^{{}^{\prime }}$$

$$s^2=\frac{1}{n-1}\sum _{i=1}^n\left(x_i-x^{{}^{\prime }}\right),x^{{}^{\prime }}=\frac{{\sum }_{i=1}^n}{x_i}$$

$$\mu_{ik}=\left\{\begin{array}[]{ll}{0}&{SF_{i}=a}\\ {\left(SF_{i}-a\right)/(a-b)}&{a<SF_{i}<b}\\ {1}&{SF_{i}=b}\end{array}\right.$$

$$S{F}_i=\sum _{j=1}^m\sqrt{{x_{ij}}^2}$$

$$d_{ij}^w=d_{ij}\ast w_j$$

$$w_{j}=\left\{\begin{array}[]{ll}{0}&{VMR_{j}=a}\\ {\left(VMR_{j}-a\right)/(a-b)}&{a<VMR_{j}<b}\\ {1}&{VMR_{j}=b}\end{array}\right.$$

$$RI=\frac{a+d}{a+b+c+d}$$

$$P=\frac{1}{n}\sum _{j=1}^c{d}_j$$

$$ER=\left(1-\frac{cp}{n}\right)\ast 100$$

Full text

††journal: Journal of Applied Computing and Informaticst1t1footnotetext: This document is a collaborative effort by Intelligent Systems Research Group Indonesia and Informatics Department Universitas Muhammadiyah Semarang.

1 Introduction

Cluster analysis or clustering is the process of partitioning a set of data objects into subset or clusters, where the objects in a cluster is similar to one another and dissimilar to the objects on other clusters [1, 2, 3]. Clustering algorithms appear as the formal tools for the computer-aided detection of the naturally occurring groups in a collection of objects or data set [4]. Clustering is the task for grouping a set of objects into a number of clusters by analyse similarity of some feature of object.

In general, clustering methods classified into four categories [2]:

Partition methods: based on distance metric between object and centroid iteratively until convergence 2. 2.

Hierarchical methods: create hierarchical decomposition of given dataset 3. 3.

Density-based methods: find clusters of arbitrary shape, dense regions in the data space, separated by sparse regions 4. 4.

Grid-based methods: quantize the object space into a finite number of cells that form a grid structure.

Hierarchical and partition methods is the most popular used clustering algorithm [5]. Compared self-organizing map neural network, FCM, k-means and traditional hierarchical clustering on 2530 dataset, FCM proved superiority and stability for all case even addition by outlier and overlapping [6]. Using UCI dataset, performance of the FCM algorithm better than k-means [7]. FCM also proving better performance over k-means for image segmentation process [8]. Several recent studies are proving the stability of the FCM algorithm over other clustering methods.

FCM is one of the most used partition methods clustering algorithm because it is naturally characteristic. FCM algorithm introduce by Dunn [9] with Fuzzy ISODATA and developed later by Bezdek [10]. Different with k-means where an object is belong to exactly one cluster, FCM allows an object belong to two or more clusters with a membership grade between zero and one [11]. Although FCM has performed well in cluster detection and proven superiority to applied in many applications, it has two interesting weakness to be improvement. The first is initialization of cluster center always set randomly, the efficiency of FCM highly depends on the initialization step, because the iterative process easily falls into a locally optimal solution [12]. The second is original FCM is used to calculate the distance with Euclidean distance, however, it proved to be worse than the weighted distance [10], The FCM algorithm is based on the Euclidean distance metric where handle each feature of the dataset with the same proportion to determine the cluster of data point. However, some of the real-world datasets have features with different contributions in determining the cluster of data point.

For many pattern recognition problems such as classification, clustering, and retrieval problems, distance metric become an importance function to define similar and dissimilarity objects [13]. By default, FCM algorithm used Euclidean distance to calculate the distance between data point and cluster centers. Another distance matric proven superiority to improve the FCM performance as Canberra distance [14, 15], city block distance [16], minkowski distance [17]. Result for the six different dataset characteristic, compared cosine, city block, chebychev and euclidean distance, canberra distance show its stability in handling various types of datasets [18]. It is clear, that used another distance metric on FCM algorithm can improve clustering performance.

To extracting meaningful information from the real-world dataset that corrupted by noise, it is not enough used distance metric only. Feature weighted is the most used task to get more improvement in clustering application. Several statistical methods like entropy [19, 20, 21] proposed to calculate feature weighted. Mean variance and standard deviation [20, 21] used to measure similarity data point and feature weighted incorporate with distance metric of the FCM algorithm. By using feature weighting, FCM performance can be better, because each feature has a different contribution in determining its class.

There have been several proposed methods in recent years to deal with FCM weaknesses in determining distances metric, primarily by combining distance function and feature weighting. In 2004, Wang et al. [22] proposed Weighted Fuzzy C-means (WFCM), by combining city block distance and feature weighting from fuzziness entropy dataset. Experimental result on iris dataset show, error rate of the WFCM algorithm about 8/150, better than FCM with 16/150. By still using entropy to calculate feature weight in Suet al. [20], proposed Entropy Weighting FCM (EWFCM). Experimental result on iris dataset, EWFCM can outperform WFCM with error rate 7/150.

In 2008, to get better feature weighting, Hung et al. [23] using bootstrapping approach and trying several fuzziness value on FCM algorithm, the best performance on iris dataset when using 10 fuzziness value with error rate 7/150, just same like EWFCM. Zhang et al. [24], in 2014 proposed fuzzy c-means clustering by genetically guided alternating optimization (GIWFCM), using genetic heuristic strategy has error rate 5/150 on iris dataset. Integrated improved feature weighting and mahalanobis distance, in 2014 Xing and Ha [25] successful minimize error rate to 5/160 just same as GIWFCM, it is called Improved feature weighting fuzzy c-means (IFWFCM).

The success of clustering process on FCM algorithm strongly influenced on the selection of exact distance metric function and the weighting of features that match the dataset characteristics. This study uses experimental method, by comparing several distance metrics to found the best performance distance measurement if applied to FCM algorithm. In addition to distance measurements, the application of some feature weightings such as the mean value, entropy, standard deviation, variance and derivatives are also comparisons to obtain appropriate weighting methods on the FCM algorithm.

In this paper, we proposed Canberra Weighted Fuzzy C-means (CWFCM). Combining Canberra distance with feature weighted used fuzziness of variance to mean ratio (VMR)[28], CWFCM has outperform with other clustering methods. To find out the improved performance of the proposed method, this study uses five public datasets from UCI Machine Learning Repository, i.e. iris, wine, Wisconsin diagnostic breast cancer (WDBC), sonar and balance. To evaluated improvement of the proposed method, we used external validation that based on historical information of the dataset [26] i.e. Rand Index RI, Purity P, Accuracy Rate AR and Error Rate ER.

The rest of the paper organized in five sections. Section 2 is the basic of theory where described foundation of the FCM algorithm, various distance metric currently used in FCM and feature weight learning. Section 3 is the proposed algorithm CWFCM. Experiment for this research descripted in section 4, where contain three sub sections i.e. dataset currently used, evaluation methods, experimental result and statistical analysis that described in tabulated. Finally, section 5 is discuses about conclusion and future work of this paper.

2 Material and algorithms

2.1 fuzzy c-means algorithm

FCM algorithm partitions a set of j-dimensional dataset $X=\left\{x_{1},x_{2},...,x_{n}\right\}$, $\left(1\leq i\leq n\right)$ into c clusters, every cluster is fuzzy set with membership degree $\mu_{ik}\left(\in\left\lfloor 0,1\right\rfloor\right)$ indicating how much the sample $x_{i}$ belong to the cluster centers $v_{j}k$. FCM aims determine cluster centers $v_{k}j\left(1\leq k\leq c\right)$, $\left(1\leq j\leq m\right)$ where m i.e. dimension of the data set and the fuzzy partition matrix $\mu_{ik}\left(1\leq k\leq c\right)$ where n i.e. sum of the data point $\left(1\leq k\leq c\right)$ by minimizing objective function as Equation (1).

[TABLE]

Where n is sum of data point, c is clusters, z is fuzziness exponent, $\mu$ is the fuzzy partition matrix that calculate with Equation (2).

[TABLE]

$d_{ij}^{2}$ is the distance between data point and cluster center, by default FCM using euclidean distance that calculate in Equation (3).

[TABLE]

Where $x_{ij}$ is the data point, and $v_{kj}$ is the cluster centers where compute as:

[TABLE]

Descripted on Figure 1, FCM algorithm consists of several stages. Stage 1 is the input of the data set, where is the data point, j aim data dimension or feature space. Stage 2 – 3 is the initialize stages, fuzziness exponent m initialized with 2, in stages 3 fuzzy partition matrix $\mu_{ik}$ is assign with random values between 0 and 1. Stage 4 is compute cluster centre $v_{kj}$ with Equation (4). Stage 5 calculate the distance using euclidean distance with Equation (3). Stage 6 compute objective function P(t) with equation 2.1 and stage 7 updating fuzzy partition matrix $\mu\left(i,k\right)$ with Equation (2), do iteratively stage 4 – 7 until stopping criterion met. Table 1 is pseudocode for FCM algorithm that descripted all stage algorithmically.

2.2 Distance matric

In the last five years, improving distance metric in FCM algorithm become interesting topic in the field of data mining. By default, FCM algorithm used Euclidean distance, another Minkowski family $\left(L_{p}\right)$ distance [12] that usually used in FCM algorithm is city block or Manhattan distance [6, 25] this distance metric defines as Equation (5).

[TABLE]

Minkowski distance also used to improve FCM performance for segmented moving object [15], it is defined as Equation (6).

[TABLE]

where p is the positive integer, when p=1 it became city block distance, when p=2 it became Euclidean distance.

In the $L_{1}$ family distance, more precisely absolute difference, the most used distance in this family on FCM algorithm is Canberra distance [27]. This distance defines as Equation (7).

[TABLE]

Moreover, another distance metric usually used in FCM algorithm is mahalanobis distance, calculate by Equation (8).

[TABLE]

where $\left(x_{i}j-v_{k}j\right)^{T}$ is the transpose matrix of $\left(x_{i}j-v_{k}j\right)$ and A is positively semi-definite, $A\geq 0.1$ [26]. When A=1, gives euclidean distance, if A is restrict to be diagonal, it is given different weight.

2.3 Feature weighted

In real world datasets, especially for the high dimensional datasets, every feature may have different degrees of relevance [28]. Feature weighting is very important task that must be have more attention during clustering process. There are several methods to decide feature weight on the datasets, that is filter methods, wrapper methods and embedded methods [29]. In this study, we using filter methods to get weight of the relevance feature from every dataset. Statistical functions to obtain suitable feature weight for each feature is variance to mean ratio [22] that calculate as:

[TABLE]

where $s^{2}$ is the sample variance of datasets and $x^{{}^{\prime}}$ is mean for each feature, it is define in Eq. ((10)).

[TABLE]

where, $x_{i}$ is the i-th data point and n is the sum of data point.

2.4 Proposed method

The proposed CWFCM method is integration of Canberra distance and feature weighted using fuzziness of VMR from each feature to determine similarity and dissimilarity data point and centroids. This method is effective to deal with the second problem of FCM algorithm i.e. distances metric.

To deal with the first problem of FCM algorithm i.e. random initialize that can make inefficient result, we used fuzziness of sum square root from each feature to initialize partition matrix. Partition matrix $\mu_{ik}$ is defined as:

[TABLE]

where, a is the smallest value and b is the largest value of SF set. SF is the sum of square root from values for each feature data point that calculate as:

[TABLE]

were, m is sum of features for each x data point.

Fig. (2) shows a flowchart of the proposed algorithm. The stage of the proposed method same is FCM algorithm stage except for stage of 3 and 5. Stage 3 is defining initialize partition matrix, defined in Eq. (11). Stage 5 calculate distance described in three sub stage i.e. calculate VMR or index of dispersion, calculate fuzziness VMR and compute metric using Canberra distance and feature weighted from fuzziness VMR. Canberra weighted distance is defined in Eq. ((13)).

[TABLE]

where, $d_{ij}$ is the Canberra distance that define by Equation (7). For better validity result, feature weight distribution appropriate in interval 0 to 1 [30]. Feature weight $w_{j}$ is fuzzy set that define as:

[TABLE]

With a being the smallest value of VMR from each feature and b is the largest value. The weighting determination using Equation (14) reduces the number of features, i.e. for features with the smallest VMR, so that, the feature with the smallest VMR value has a zero weight and features with the largest VMR value has a weight of one.

Table 2, shows a description of the pseudocode algorithmically for all stages in a flowchart of the proposed method in Fig. (2)

3 Experimental results and analysis

3.1 Datasets

Many studies by distance metric learning on the FCM algorithm are using public datasets. In this study, public datasets also used to test the proposed method. Public datasets mostly used to this study were available in UCI machine learning repository. Table (3) shows description of the datasets, from the left column is name of the datasets, number of instances, number of attributes, number of class and data distribution for each class. From the UCI machine learning repository, the most widely used datasets are selected and have varying dimensions and number of classes, so that can represent all conditions in testing the stability of the proposed method.

3.2 Evaluation

In general, there are two types of evaluation for the clustering algorithm, i.e. external validation by utilizing information on the historical data and internal validation by simply utilizing the intrinsic information of the data itself [25]. To know the proposed performance improvement, this research using external validation method, that is rand index [31], purity [32], and error rate ER.

Rand index (RI) has a range of values between zero and one, the highest value indicates better performance, this evaluation described as:

[TABLE]

For a set of n object $X=\left\{x_{1},x_{2},\ldots,x_{n}\right\}$, if $Y=\left\{y_{1},y_{2},\ldots,y_{k}\right\}$ is result of the clustering method and $Z=\left\{z_{1},z_{2},\ldots,z_{k}\right\}$ is the actual class. The a is number of pairs X were in the same subset Y and same subset in Z, b = number of pairs X were in the different subset Y and different subset in Z. The c is a number of pairs X were in the same subset Y and different subset in Z. In addition, d is a number of pairs X were in the different subset Y and same subset in Z.

Purity P focuses on the frequency of the most common category into each cluster [32] and define as:

[TABLE]

where n is sum of the data point, c is sum of clusters and $d_{j}$ is sum of data point that classified exactly on the j cluster.

Error rate ER is defined as:

[TABLE]

where, cp is the wrong classified data point and n is the sum of data point.

To evaluated efficiency for proposed method, this study using iteration that explained how many time algorithms looping until stopping criterion meet.

3.3 Experiment results

The computer specifications used to simulate the performance of the proposed algorithm descripted on Table (4). The proposed algorithm is implement using basic language on Ms. Visual Studio 2012. The add noise function on RapidMiner 7.5 is used to generate additional noise on the datasets.

Table (5) showing comparison of proposed method with other FCM methods on the iris dataset. The second column is feature weight for each algorithm except Bezdek [10] or native methods were not present by feature weighted. The third column is percentage of error rates for each algorithm and the fourth column is data point that misclassification for each algorithm.

Still using iris dataset, Table (6) showing comparison several distance matric with three feature weighted on FCM algorithm. The first column descripted three kind of feature weighted, second column is additional noise from 0 to 30 for each feature weighted and the third column until eighth is distance metric i.e. Euclidean (Eucl.), City Block (CB), Canberra (Canb.), Minkowsky (Mink.) and Mahalanobis (Maha.) distance. The first row for every grouping is iteration that consume by the methods, for left side by using random initialize and the right one is using proposed initialize. The second row for every grouping is percentage of error rates for the methods.

As you can see in Table (6), the proposed initialization is always superior to random initialization and smallest iteration on every dataset is when using average entropy and Canberra distance with 5, 5, 4, 8 and 8. The best performance produces error rates for all dataset is when using VMR and Canberra distance, except for 30$\%$ additional noise, average gain with Euclidean and Mahalanobis distance difference of 1.33, better than proposed method with 8.00$\%$ error rates.

With using VMR feature weighted as descripted on Table (6) and Table (7), canberra distance shows the best performance on each dataset. The proposed algorithm shows its stability to handle both datasets even with several additional noise. The proposed method also proven efficiency with iteration between 7 and 10 for iris and 13 until 16 for WDBC. This proves that the use of feature weights using VMR on canberra distance is very influential on the success of the clustering process.

To describe superiority of the proposed initialization method, Table (8) is a comparison of the results of the average number of iterations obtained when using VMR feature weighting and canberra distance as outlined in Table 6 and Table (7). Can be seen that, the average initialization proposed in this study has fewer iterations than all random models, this proves from the side of computational time, the proposed method is more efficient than the previous method.

Just like IFWFCM, CWFCM improvised FCM stages by changing the initialization process, features weighted and changing euclidean distances using canberra distance. To test the stability of CWFCM, Table (9), shows comparison of the proposed method with the previous method in the terms of computational cost. This study uses average of iterations and time in seconds that’s needed until the stopping criterion of the algorithm are met or the partition is declared optimal. Each method is run 10 times in each dataset, then the average value of the iteration results and the computational times in seconds is taken to obtain an objective comparison of the scores.

As can be seen in Table (9), the number of iterations is not always directly proportional to the time needed to reach the optimal partition. The most significant difference is seen in the abalone dataset, the average number of FCM iterations is 33.6 while the IFWFCM iteration average is smaller at 26.5, but FCM takes less time i.e. 13.133 seconds while IFWFCM takes 103,041 seconds, more than 1 minute to achieve optimal conditions, in this research, this is the longest time ever. In the same dataset CWFCM only requires 21 iterations with 8.19 seconds needed.

Judging from the time needed to meet optimum conditions, CWFCM is only longer on the WDBC and WBC datasets. In the WDBC dataset the fastest time is IFWFCM with 1.532 seconds, 0.179 seconds faster than CWFCM. As for the WBC dataset, the fastest method is FCM with 0.632 seconds, 0.705 seconds faster than CWFCM with 1,337 secondson average CWFCM outperforms other methods with 1.568 second, while FCM 2.551 seconds and IFWFCM takes an average of 13.180 seconds to reach optimal conditions.As you can see from the results of the table explanation, that CWFCM can be stated to have more efficient computational costs.

The computational costs required by each method described in Table 9 cannot be stated as the only measure that a method with minimum costs is at least superior to a method that has more computational costs. To be more convincing about superiority of the proposed method, Table 10 presents the results of a comparison of methods using various external validation. In the term of Purity, CWFCM has the best performance in all datasets, even though IFWFCM has the same results as CWFCM on 3 datasets namely WDBC, Wholesale Customers and Bupa, as well as FCM which has the same results on the Bupa dataset. On average CWFCM obtained a purity value of 0.705, superior to 0.011 from IFWFCM and outperformed 0.049 from FCM.

In addition, to further prove the superiority of the proposed method, although it is not as perfect as using Purity, with using Rand Index, from 9 datasets, CWFCM excels at 5 datasets, while IFWFCM excels in 3 datasets i.e. WDBC, Yeast and Bupa, while FCM excels in only one dataset i.e. TAE. However, on average even though it has a difference of only 0.001 from IFWFCM, CWFCM still excels with an average rand index of 0.619, and superior to FCM with a difference of 0.080.

Finally, to determine the level of cluster error, this study uses the error rates. Error rates are displayed in the rightmost column group, error rates can be used to find out the number of instances that can’t be correctly classified or as mentioned misclassification (MC) on Table 5.Number of MC are calculated by equation:error rates * total number of instant / 100, for example in the iris dataset the number of instant that is not properly classified is 4 instant, obtained from 2,667*150/100. In line with the results of purity, CWFCM shows very satisfactory results, with fewer error rates in all datasets outperforming IFWFCM or FCM.

3.4 Statistical Analysis

To compare several methods on multiple datasets, this study uses a non-parametric statistical test, i.e. the Friedman test to decide differences in clustering problems. To get comprehensive and convincing results, the statistical analysis process uses the help of XLSTAT software. Table 11. shows the results of Friedman’s test for the evaluation method of error rates, from the table can explain that Friedman’s test produces a p-value of 0.0035, smaller than the alpha value of 0.05, concluded for this condition is there are significant differences between methods.

To find out a comparison of which methods have significant differences, we use Nemenyi’s post-hoc test so that the difference between method is clearly seen.A significant difference is if the p-values between the two methods are smaller than the 0.05 alpha value. In Table 12. it can be seen, the p-values between CWFCM and FCM are 0.0090 ¡ 0.05 and the p-values between CWFCM and IFWFCM are 0.0483 ¡0.05.It is clear that the proposed method CWFCM has significant differences both with FCM and IFWFCM.

In line with state-of-the-art of this study [14, 15, 16], that other distance metric is possible for using as distance metric in the FCM algorithm to produce better clustering. Evident in this study, that canberra distance has a good level of stability to decide the best distance between the data points and the cluster centers. This study also confirms the findings of previous research, that each feature will have a different weight in determining its class [20, 23, 25], it is proven by adding feature weights using the VMR, proposed method shows a very satisfying performance even in noisy datasets, multidimensional feature and have a lot of classes.

4 Conclusions and feature works

There are two major drawbacks that have been discovered by previous researchers, to improve performance of the FCM algorithm. In the terms of efficiency, random initialization is the first drawback, in terms of the accuracy of grouping data euclidean distance becomes the second drawback because it treats all features with the same weight. In this study, we propose a novel method that we called CWFCM, we used fuzziness of sum square root from each feature to initialize partition matrix and replaced the initialization process randomly. To overcome the second drawback, we propose by adding VMR to get feature weights that match the characteristics of each datasets and change the euclidean distance by canberra distance to get the optimum distance between data point with each cluster center. We have compared several distance measurements and feature weights that are often used in the FCM algorithm, on average the proposed method outperform even with the added noise up to 30$\%$.In addition, we have also compared the proposed method with the earlier method using various measuring instruments, and the results of proposed method outperform both in terms of computational time and accuracy in grouping data. The results of statistical tests using non-parametric statistical tests also show that the proposed method has a significant difference compared to the earlier method.

Clustering is the unsupervised learning that can predict the number of classes based on observation methods from the characteristics of the datasets. Furthermore its will be interesting to add an internal validation model that evaluated in the terms of automatically determining the number of clusters in the FCM algorithm.