LMFAO: An Engine for Batches of Group-By Aggregates

TL;DR

LMFAO is an in-memory engine optimized for efficiently executing large batches of group-by aggregate queries over joins, significantly accelerating data science workloads such as regression, decision trees, and clustering.

Contribution

The paper introduces LMFAO, a novel in-memory engine specifically designed for batch group-by aggregates over joins, improving performance for data science tasks.

Findings

Demonstrates LMFAO's effectiveness on ridge linear regression, decision trees, and clustering.

Shows significant speedups over traditional database engines.

Validates LMFAO's applicability to real-world data science models.

Abstract

LMFAO is an in-memory optimization and execution engine for large batches of group-by aggregates over joins. Such database workloads capture the data-intensive computation of a variety of data science applications. We demonstrate LMFAO for three popular models: ridge linear regression with batch gradient descent, decision trees with CART, and clustering with Rk-means.