Two-Chains: High Performance Framework for Function Injection and Execution

TL;DR

Two-Chains is a high-performance framework enabling dynamic, low-latency remote function injection and execution on large-scale distributed systems, leveraging RDMA and binary manipulation for efficient task migration.

Contribution

It introduces a novel framework that combines binary-level function injection with RDMA for ultra-low latency distributed execution in user space.

Findings

Stashing reduces latency and increases message rates.

Framework achieves HPC-level performance for function injection.

Seamless interoperability with existing C libraries.

Abstract

Some important problems, such as semantic graph analysis, require large-scale irregular applications composed of many coordinating tasks that operate on a shared data set so big it has to be stored on many physical devices. In these cases, it may be more efficient to dynamically choose where code runs as the applications progresses. Many programming environments provide task migration or remote function calls, but they have sharp trade-offs between flexible composition, portability, performance, and code complexity. We developed Two-Chains, a high performance framework inspired by active message communication semantics. We use the GNU Binutils, the ELF binary format, and the RDMA network protocol to provide ultra-low granularity distributed function composition at runtime in user space at HPC performance levels using C libraries. Our framework allows the direct injection of function binaries and data to a remote machine cache using the RDMA network. It interoperates seamlessly with existing C libraries using standard dynamic linking and load symbol resolution. We analyze function delivery and execution on cache stashing-enabled hardware and show that stashing decreases latency, increases message rates, and improves noise tolerance. This demonstrates one way this method is suited to increasingly network-oriented hardware architectures.