CRAC: Checkpoint-Restart Architecture for CUDA with Streams and UVM

TL;DR

CRAC is a scalable, low-overhead checkpoint-restart architecture for CUDA applications on NVIDIA GPUs, supporting full CUDA features and enabling fault tolerance in supercomputing environments.

Contribution

It introduces a novel, efficient checkpoint-restart solution for CUDA that supports streams and UVM, reducing overhead and simplifying fault tolerance implementation.

Findings

Approximately 1% runtime overhead

Fast checkpoint-restart times

Supports scalable CUDA streams and UVM

Abstract

The share of the top 500 supercomputers with NVIDIA GPUs is now over 25% and continues to grow. While fault tolerance is a critical issue for supercomputing, there does not currently exist an efficient, scalable solution for CUDA applications on NVIDIA GPUs. CRAC (Checkpoint-Restart Architecture for CUDA) is new checkpoint-restart solution for fault tolerance that supports the full range of CUDA applications. CRAC combines: low runtime overhead (approximately 1% or less); fast checkpoint-restart; support for scalable CUDA streams (for efficient usage of all of the thousands of GPU cores); and support for the full features of Unified Virtual Memory (eliminating the programmer's burden of migrating memory between device and host). CRAC achieves its flexible architecture by segregating application code (checkpointed) and its external GPU communication via non-reentrant CUDA libraries (not checkpointed) within a single process's memory. This eliminates the high overhead of inter-process communication in earlier approaches, and has fewer limitations.