Bring the BitCODE -- Moving Compute and Data in Distributed Heterogeneous Systems

TL;DR

This paper introduces a framework that enables moving compute and data across distributed heterogeneous systems using LLVM and UCX, supporting dynamic code optimization, propagation, and integration with high-level languages like Julia.

Contribution

It presents a novel architecture and implementation for distributed heterogeneous computing, including a new class of X-RDMA operations and integration with modern programming languages.

Findings

X-RDMA pointer chase outperforms RDMA GET by 70%

Framework supports dynamic code propagation and optimization

Enables high-performance distributed computing in heterogeneous systems

Abstract

In this paper, we present a framework for moving compute and data between processing elements in a distributed heterogeneous system. The implementation of the framework is based on the LLVM compiler toolchain combined with the UCX communication framework. The framework can generate binary machine code or LLVM bitcode for multiple CPU architectures and move the code to remote machines while dynamically optimizing and linking the code on the target platform. The remotely injected code can recursively propagate itself to other remote machines or generate new code. The goal of this paper is threefold: (a) to present an architecture and implementation of the framework that provides essential infrastructure to program a new class of disaggregated systems wherein heterogeneous programming elements such as compute nodes and data processing units (DPUs) are distributed across the system, (b) to demonstrate how the framework can be integrated with modern, high-level programming languages such as Julia, and (c) to demonstrate and evaluate a new class of eXtended Remote Direct Memory Access (X-RDMA) communication operations that are enabled by this framework. To evaluate the capabilities of the framework, we used a cluster with Fujitsu CPUs and heterogeneous cluster with Intel CPUs and BlueField-2 DPUs interconnected using high-performance RDMA fabric. We demonstrated an X-RDMA pointer chase application that outperforms an RDMA GET-based implementation by 70% and is as fast as Active Messages, but does not require function predeployment on remote platforms.