Phantora: Maximizing Code Reuse in Simulation-based Machine Learning System Performance Estimation

TL;DR

Phantora is a hybrid GPU cluster simulator that enables high-fidelity performance estimation of ML training workloads by executing unmodified ML frameworks within a containerized environment, reducing manual effort and maintaining accuracy.

Contribution

It introduces a hybrid simulation approach that reuses actual ML framework code, simplifying performance estimation without reimplementation.

Findings

Provides accuracy comparable to static workload simulation.

Supports multiple state-of-the-art LLM training frameworks out-of-the-box.

Operates on a single GPU, reducing resource requirements.

Abstract

Modern machine learning (ML) training workloads place substantial demands on both computational and communication resources. Consequently, accurate performance estimation has become increasingly critical for guiding system design decisions, such as the selection of parallelization strategies, cluster configurations, and hardware provisioning. Existing simulation-based performance estimation requires reimplementing the ML framework in a simulator, which demands significant manual effort and is hard to maintain as ML frameworks evolve rapidly. This paper introduces Phantora, a hybrid GPU cluster simulator designed for performance estimation of ML training workloads. Phantora executes unmodified ML frameworks as is within a distributed, containerized environment. Each container emulates the behavior of a GPU server in a large-scale cluster, while Phantora intercepts and simulates GPU- and communication-related operations to provide high-fidelity performance estimation. We call this approach hybrid simulation of ML systems, in contrast to traditional methods that simulate static workloads. The primary advantage of hybrid simulation is that it allows direct reuse of ML framework source code in simulation, avoiding the need for reimplementation. Our evaluation shows that Phantora provides accuracy comparable to static workload simulation while supporting three state-of-the-art LLM training frameworks out-of-the-box. In addition, Phantora operates on a single GPU, eliminating the need for the resource-intensive trace collection and workload extraction steps required by traditional trace-based simulators. Phantora is open-sourced at https://github.com/QDelta/Phantora.