Ilargi: a GPU Compatible Factorized ML Model Training Framework

TL;DR

Ilargi is a pioneering GPU-compatible factorized ML training framework that automates data integration and optimizes computation, significantly accelerating training across CPU and GPU environments with intelligent cost-based decision making.

Contribution

Introduces Ilargi, the first GPU-compatible factorized learning framework that automates data integration and employs ML-based cost estimation for optimized training.

Findings

Achieves up to 8.9x speedups on GPUs.

Over 20% acceleration in batch ML workloads.

First to enable GPU-compatible factorized learning.

Abstract

The machine learning (ML) training over disparate data sources traditionally involves materialization, which can impose substantial time and space overhead due to data movement and replication. Factorized learning, which leverages direct computation on disparate sources through linear algebra (LA) rewriting, has emerged as a viable alternative to improve computational efficiency. However, the adaptation of factorized learning to leverage the full capabilities of modern LA-friendly hardware like GPUs has been limited, often requiring manual intervention for algorithm compatibility. This paper introduces Ilargi, a novel factorized learning framework that utilizes matrix-represented data integration (DI) metadata to facilitate automatic factorization across CPU and GPU environments without the need for costly relational joins. Ilargi incorporates an ML-based cost estimator to intelligently selects between factorization and materialization based on data properties, algorithm complexity, hardware environments, and their interactions. This strategy ensures up to 8.9x speedups on GPUs and achieves over 20% acceleration in batch ML training workloads, thereby enhancing the practicability of ML training across diverse data integration scenarios and hardware platforms. To our knowledge, this work is the very first effort in GPU-compatible factorized learning.