PystachIO: Efficient Distributed GPU Query Processing with PyTorch over Fast Networks & Fast Storage

TL;DR

PystachIO is a novel distributed GPU query engine built on PyTorch that optimizes network and storage I/O to efficiently handle large-scale OLAP workloads, achieving significant speedups.

Contribution

It introduces PystachIO, a PyTorch-based distributed OLAP engine with optimizations for maximizing GPU, network, and storage utilization in large-scale environments.

Findings

Up to 3x end-to-end speedups over existing approaches.

Effective overlapping of computation and data movement improves utilization.

Supports scalable, storage-resident OLAP workloads.

Abstract

The AI hardware boom has led modern data centers to adopt HPC-style architectures centered on distributed, GPU-centric computation. Large GPU clusters interconnected by fast RDMA networks and backed by high-bandwidth NVMe storage enable scalable computation and rapid access to storage-resident data. Tensor computation runtimes (TCRs), such as PyTorch, originally designed for AI workloads, have recently been shown to accelerate analytical workloads. However, prior work has primarily considered settings where the data fits in aggregated GPU memory. In this paper, we systematically study how TCRs can support scalable, distributed query processing for large-scale, storage-resident OLAP workloads. Although TCRs provide abstractions for network and storage I/O, naive use often underutilizes GPU and I/O bandwidth due to insufficient overlap between computation and data movement. As a core contribution, we present PystachIO, a prototype of a PyTorch-based distributed OLAP engine that combines fast network and storage I/O with key optimizations to maximize GPU, network, and storage utilization. Our evaluation shows up to 3x end-to-end speedups over existing distributed GPU-based query processing approaches.