clusterNOR: A NUMA-Optimized Clustering Framework

TL;DR

clusterNOR is a flexible, NUMA-optimized clustering framework that significantly accelerates various clustering algorithms by reducing memory bottlenecks and barriers, enabling scalable high-performance clustering on large datasets.

Contribution

It introduces a generic, extensible clustering framework optimized for NUMA architectures, with novel algorithms and optimizations that outperform existing solutions.

Findings

Order of magnitude speedup over state-of-the-art solutions

Supports in-memory, semi-external, and distributed execution

Efficient pruning algorithm for billion-point datasets

Abstract

Clustering algorithms are iterative and have complex data access patterns that result in many small random memory accesses. The performance of parallel implementations suffer from synchronous barriers for each iteration and skewed workloads. We rethink the parallelization of clustering for modern non-uniform memory architectures (NUMA) to maximizes independent, asynchronous computation. We eliminate many barriers, reduce remote memory accesses, and maximize cache reuse. We implement the 'Clustering NUMA Optimized Routines' (clusterNOR) extensible parallel framework that provides algorithmic building blocks. The system is generic, we demonstrate nine modern clustering algorithms that have simple implementations. clusterNOR includes (i) in-memory, (ii) semi-external memory, and (iii) distributed memory execution, enabling computation for varying memory and hardware budgets. For algorithms that rely on Euclidean distance, clusterNOR defines an updated Elkan's triangle inequality pruning algorithm that uses asymptotically less memory so that it works on billion-point data sets. clusterNOR extends and expands the scope of the 'knor' library for k-means clustering by generalizing underlying principles, providing a uniform programming interface and expanding the scope to hierarchical and linear algebraic classes of algorithms. The compound effect of our optimizations is an order of magnitude improvement in speed over other state-of-the-art solutions, such as Spark's MLlib and Apple's Turi.