Preparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca

TL;DR

Seneca is a data loading system that optimizes cache partitioning and data sampling to significantly reduce training time and improve throughput for multimedia ML models.

Contribution

It introduces a novel cache partitioning and sampling approach that enhances data pipeline efficiency during concurrent ML training.

Findings

Reduces training makespan by 45.23% compared to PyTorch.

Increases data processing throughput by up to 3.45x.

Outperforms state-of-the-art caching systems in experiments.

Abstract

Input data preprocessing is a common bottleneck when concurrently training multimedia machine learning (ML) models in modern systems. To alleviate these bottlenecks and reduce the training time for concurrent jobs, we present Seneca, a data loading system that optimizes cache partitioning and data sampling for the data storage and ingestion (DSI) pipeline. The design of Seneca contains two key techniques. First, Seneca uses a performance model for the data pipeline to optimally partition the cache for three different forms of data (encoded, decoded, and augmented). Second, Seneca opportunistically serves cached data over uncached ones during random batch sampling so that concurrent jobs benefit from each other. We implement Seneca by modifying PyTorch and demonstrate its effectiveness by comparing it against several state-of-the-art caching systems for DNN training. Seneca reduces the makespan by 45.23% compared to PyTorch and increases data processing throughput by up to 3.45x compared to the next best dataloader.