Efficient Large Scale Clustering based on Data Partitioning
Malika Bendechache, Nhien-An Le-Khac, M-Tahar Kechadi
TL;DR
This paper introduces a scalable distributed clustering method that efficiently handles large datasets by combining local clustering with an effective aggregation process, producing accurate global clusters without predefining their number.
Contribution
It presents a novel distributed clustering approach that improves efficiency and accuracy in global model generation, adaptable to different local clustering techniques.
Findings
The method is scalable to large datasets.
It produces high-quality, accurate global clusters.
The number of global clusters is dynamically determined.
Abstract
Clustering techniques are very attractive for extracting and identifying patterns in datasets. However, their application to very large spatial datasets presents numerous challenges such as high-dimensionality data, heterogeneity, and high complexity of some algorithms. For instance, some algorithms may have linear complexity but they require the domain knowledge in order to determine their input parameters. Distributed clustering techniques constitute a very good alternative to the big data challenges (e.g.,Volume, Variety, Veracity, and Velocity). Usually these techniques consist of two phases. The first phase generates local models or patterns and the second one tends to aggregate the local results to obtain global models. While the first phase can be executed in parallel on each site and, therefore, efficient, the aggregation phase is complex, time consuming and may produce…
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8 4_Archi-DBSCAN.png)
Figure 9 4_Archi-Kmeans.png)
Figure 10 4_D1.png)
Figure 11 4_D2.png)
Figure 12 4_D2CA.png)
Figure 13 4_DBSCANresult.png)
Figure 14 4_DistArchi.png)
Figure 15 4_ET.png)
Figure 16 4_IEEEtran_HOWTO.jpg)
Figure 17 4_KMEANS.png)
Figure 18 4_KmeansRE.png)
Figure 19 4_SCALUP.png)
Figure 20 4_SCALUP1.png)
Figure 21 4_T3-DBSCAN.png)
Figure 22 4_finalresult.png)
Figure 23 4_images2_Birch1.png)
Figure 24 4_images2_Birch2.png)
Figure 25 4_images2_Birch3.png)
Figure 26 4_images2_Birch4.png)
Figure 27 4_images2_Birch5.png)
Figure 28 4_images2_Birch6.png)
Figure 29 4_images2_Cure1.png)
Figure 30 4_images2_Cure2.png)
Figure 31 4_images2_Cure3.png)
Figure 32 4_images2_Cure4.png)
Figure 33 4_images2_Cure5.png)
Figure 34 4_images2_Cure6.png)
Figure 35 4_images2_D2CA1.png)
Figure 36 4_images2_D2CA2.png)
Figure 37 4_images2_D2CA3.png)
Figure 38 4_images2_D2CA4.png)
Figure 39 4_images2_D2CA5.png)
Figure 40Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.