Ambiguity-Driven Fuzzy C-Means Clustering: How to Detect Uncertain Clustered Records

TL;DR

This paper introduces a certainty factor into Fuzzy C-Means clustering to identify ambiguous records, reducing false detections and improving accuracy in critical decision-making applications.

Contribution

It proposes a novel method to detect uncertain records in FCM by incorporating a certainty factor, enhancing accuracy without sacrificing performance.

Findings

Significant decrease in error rate across multiple datasets

Improved sensitivity of the clustering algorithm

Maintains low processing time for most records

Abstract

As a well-known clustering algorithm, Fuzzy C-Means (FCM) allows each input sample to belong to more than one cluster, providing more flexibility than non-fuzzy clustering methods. However, the accuracy of FCM is subject to false detections caused by noisy records, weak feature selection and low certainty of the algorithm in some cases. The false detections are very important in some decision-making application domains like network security and medical diagnosis, where weak decisions based on such false detections may lead to catastrophic outcomes. They are mainly emerged from making decisions about a subset of records that do not provide enough evidence to make a good decision. In this paper, we propose a method for detecting such ambiguous records in FCM by introducing a certainty factor to decrease invalid detections. This approach enables us to send the detected ambiguous records to another discrimination method for a deeper investigation, thus increasing the accuracy by lowering the error rate. Most of the records are still processed quickly and with low error rate which prevents performance loss compared to similar hybrid methods. Experimental results of applying the proposed method on several datasets from different domains show a significant decrease in error rate as well as improved sensitivity of the algorithm.