PhoenixOS: Concurrent OS-level GPU Checkpoint and Restore with Validated Speculation

TL;DR

PHOENIXOS introduces a novel OS service enabling concurrent checkpoint and restore of GPU processes, addressing unique GPU challenges with speculative and validated memory access detection for fault tolerance and process migration.

Contribution

It is the first to enable concurrent GPU process checkpointing and restoring at the OS level using speculative validation techniques, improving performance and reliability.

Findings

Orders of magnitude higher performance than NVIDIA cuda-checkpoint.

Effective GPU memory access detection through speculation and validation.

Supports fault tolerance, process migration, and fast startup for GPU workloads.

Abstract

PHOENIXOS (PHOS) is the first OS service that can concurrently checkpoint and restore (C/R) GPU processes--a fundamental capability for critical tasks such as fault tolerance, process migration, and fast startup. While concurrent C/R is well-established on CPUs, it poses unique challenges on GPUs due to their lack of essential features for efficiently tracing concurrent memory reads and writes, such as specific hardware capabilities (e.g., dirty bits) and OS-mediated data paths (e.g., copy-on-write). To ensure correct concurrent C/R, PHOS proactively detects GPU memory reads and writes through a two-step process: first, it speculates about GPU memory accesses based on the arguments used when launching GPU kernels; then, it validates these accesses efficiently at runtime using binary instrumentation. With this validated speculation, PHOS retrofits CPU-based concurrent C/R for GPUs through software-based approaches, including soft copy-on-write, soft recopy, and soft on-demand restore. PHOS further proposes several GPU-aware techniques for efficient GPU C/R, including coordinated checkpoint data transfer and execution context pool. For downstream tasks that use C/R for tolerating failures, migrating processes between machines, and accelerating cold starts in serverless computing, PHOS achieves orders of magnitude higher performance than state-of-the-art OS-level GPU C/R systems like NVIDIA cuda-checkpoint.