RHAPSODY: Execution of Hybrid AI-HPC Workflows at Scale

TL;DR

RHAPSODY is a middleware that enables seamless, scalable execution of complex hybrid AI-HPC workflows by coordinating multiple runtimes, supporting diverse workloads, and maintaining high performance on large HPC systems.

Contribution

It introduces a multi-runtime middleware that composes and coordinates existing runtimes to support heterogeneous AI-HPC workflows at scale.

Findings

Achieves near-linear scaling for inference workloads

Introduces minimal runtime overhead

Supports heterogeneous AI-HPC workflows efficiently

Abstract

Hybrid AI-HPC workflows combine large-scale simulation, training, high-throughput inference, and tightly coupled, agent-driven control within a single execution campaign. These workflows impose heterogeneous and often conflicting requirements on runtime systems, spanning MPI executables, persistent AI services, fine-grained tasks, and low-latency AI-HPC coupling. Existing systems typically address only subsets of these requirements, limiting their ability to support emerging AI-HPC applications at scale. We present RHAPSODY, a multi-runtime middleware that enables concurrent execution of heterogeneous AI-HPC workloads through uniform abstractions for tasks, services, resources, and execution policies. Rather than replacing existing runtimes, RHAPSODY composes and coordinates them, allowing simulation codes, inference services, and agentic workflows to coexist within a single job allocation on leadership-class HPC platforms. We evaluate RHAPSODY with Dragon and vLLM on multiple HPC systems using representative heterogeneous, inference-at-scale, and tightly coupled AI-HPC workflows. Our results show that RHAPSODY introduces minimal runtime overhead, sustains increasing heterogeneity at scale, achieves near-linear scaling for high-throughput inference workloads, and data- and control-efficient coupling between AI and HPC tasks in agentic workflows.