GraphLab: A New Framework For Parallel Machine Learning

TL;DR

GraphLab is a novel parallel framework tailored for machine learning that efficiently expresses asynchronous algorithms with sparse dependencies, enabling high-performance execution on large-scale problems.

Contribution

It introduces GraphLab, a high-level abstraction that simplifies designing parallel ML algorithms with data consistency and performance guarantees.

Findings

Successfully implemented parallel belief propagation, Gibbs sampling, Co-EM, Lasso, and Compressed Sensing.

Achieved high parallel performance on large-scale real-world datasets.

Demonstrated expressiveness and efficiency of GraphLab for diverse ML algorithms.

Abstract

Designing and implementing efficient, provably correct parallel machine learning (ML) algorithms is challenging. Existing high-level parallel abstractions like MapReduce are insufficiently expressive while low-level tools like MPI and Pthreads leave ML experts repeatedly solving the same design challenges. By targeting common patterns in ML, we developed GraphLab, which improves upon abstractions like MapReduce by compactly expressing asynchronous iterative algorithms with sparse computational dependencies while ensuring data consistency and achieving a high degree of parallel performance. We demonstrate the expressiveness of the GraphLab framework by designing and implementing parallel versions of belief propagation, Gibbs sampling, Co-EM, Lasso and Compressed Sensing. We show that using GraphLab we can achieve excellent parallel performance on large scale real-world problems.