UCX Programming Interface for Remote Function Injection and Invocation

TL;DR

This paper introduces a novel UCX-based API enabling remote function injection and invocation, facilitating dynamic code dispatch across diverse hardware like SmartNICs and remote servers for large-scale, irregular applications.

Contribution

The work presents a new API that extends network communication capabilities to include remote code execution, overcoming traditional SPMD constraints and supporting dynamic, large-scale application workflows.

Findings

Prototype implementation demonstrates fast compute migration via RDMA.

API supports dispatching user functions to various hardware including SmartNICs.

Potential for improved efficiency in large-scale irregular applications.

Abstract

Network library APIs have historically been developed with the emphasis on data movement, placement, and communication semantics. Many communication semantics are available across a large variety of network libraries, such as send-receive, data streaming, put/get/atomic, RPC, active messages, collective communication, etc. In this work we introduce new compute and data movement APIs that overcome the constraints of the single-program, multiple-data (SPMD) programming model by allowing users to send binary executable code between processing elements. Our proof-of-concept implementation of the API is based on the UCX communication framework and leverages the RDMA network for fast compute migration. We envision the API being used to dispatch user functions from a host CPU to a SmartNIC (DPU), computational storage drive (CSD), or remote servers. In addition, the API can be used by large-scale irregular applications (such as semantic graph analysis), composed of many coordinating tasks operating on a data set so big that it has to be stored on many physical devices. In such cases, it may be more efficient to dynamically choose where code runs as the applications progresses.