COX: CUDA on X86 by Exposing Warp-Level Functions to CPUs

TL;DR

COX is a framework that enables efficient execution of modern CUDA programs on CPU platforms by exposing warp-level functions to CPUs, achieving high coverage and performance.

Contribution

The paper introduces hierarchical collapsing and a new LLVM pass to support CUDA warp-level functions on CPUs, significantly improving compatibility and efficiency.

Findings

Supports 90% of CUDA features compared to 68% in previous frameworks

Efficient execution of warp-level functions using CPU SIMD (AVX) instructions

High application coverage with comparable performance to GPU execution

Abstract

As CUDA programs become the de facto program among data parallel applications such as high-performance computing or machine learning applications, running CUDA on other platforms has been a compelling option. Although several efforts have attempted to support CUDA on other than NVIDIA GPU devices, due to extra steps in the translation, the support is always behind a few years from supporting CUDA's latest features. The examples are DPC, Hipfy, where CUDA source code have to be translated to their native supporting language and then they are supported. In particular, the new CUDA programming model exposes the warp concept in the programming language, which greatly changes the way the CUDA code should be mapped to CPU programs. In this paper, hierarchical collapsing that \emph{correctly} supports CUDA warp-level functions on CPUs is proposed. Based on hierarchical collapsing, a framework, COX, is developed that allows CUDA programs with the latest features to be executed efficiently on CPU platforms. COX consists of a compiler IR transformation (new LLVM pass) and a runtime system to execute the transformed programs on CPU devices. COX can support the most recent CUDA features, and the application coverage is much higher (90%) than for previous frameworks (68%) with comparable performance. We also show that the warp-level functions in CUDA can be efficiently executed by utilizing CPU SIMD (AVX) instructions.