M3: Scaling Up Machine Learning via Memory Mapping

TL;DR

This paper introduces M3, a memory mapping-based approach that enables scalable, out-of-core machine learning on large datasets, achieving performance comparable to larger distributed clusters without the need for distributed computing.

Contribution

We demonstrate that memory mapping can effectively scale general machine learning algorithms like logistic regression and k-means on datasets up to 190GB, offering a faster alternative to distributed systems.

Findings

Memory mapping is feasible for large-scale machine learning.

M3 outperforms a 4-instance Spark cluster in speed.

M3 matches the performance of an 8-instance Spark cluster.

Abstract

To process data that do not fit in RAM, conventional wisdom would suggest using distributed approaches. However, recent research has demonstrated virtual memory's strong potential in scaling up graph mining algorithms on a single machine. We propose to use a similar approach for general machine learning. We contribute: (1) our latest finding that memory mapping is also a feasible technique for scaling up general machine learning algorithms like logistic regression and k-means, when data fits in or exceeds RAM (we tested datasets up to 190GB); (2) an approach, called M3, that enables existing machine learning algorithms to work with out-of-core datasets through memory mapping, achieving a speed that is significantly faster than a 4-instance Spark cluster, and comparable to an 8-instance cluster.