CoroAMU: Unleashing Memory-Driven Coroutines through Latency-Aware Decoupled Operations

TL;DR

CoroAMU is a hardware-software co-designed system that enhances memory-driven coroutines to better hide memory latency in data-intensive applications, achieving significant performance improvements on FPGA and server platforms.

Contribution

It introduces compiler optimizations and hardware support for decoupled memory operations, improving coroutine efficiency and latency hiding in disaggregated memory systems.

Findings

Achieves 1.51x speedup over state-of-the-art coroutine methods.

Delivers up to 4.87x performance improvements on FPGA-emulated systems.

Effectively balances latency hiding with runtime overhead.

Abstract

Modern data-intensive applications face memory latency challenges exacerbated by disaggregated memory systems. Recent work shows that coroutines are promising in effectively interleaving tasks and hiding memory latency, but they struggle to balance latency-hiding efficiency with runtime overhead. We present CoroAMU, a hardware-software co-designed system for memory-centric coroutines. It introduces compiler procedures that optimize coroutine code generation, minimize context, and coalesce requests, paired with a simple interface. With hardware support of decoupled memory operations, we enhance the Asynchronous Memory Unit to further exploit dynamic coroutine schedulers by coroutine-specific memory operations and a novel memory-guided branch prediction mechanism. It is implemented with LLVM and open-source XiangShan RISC-V processor over the FPGA platform. Experiments demonstrate that the CoroAMU compiler achieves a 1.51x speedup over state-of-the-art coroutine methods on Intel server processors. When combined with optimized hardware of decoupled memory access, it delivers 3.39x and 4.87x average performance improvements over the baseline processor on FPGA-emulated disaggregated systems under 200ns and 800ns latency respectively.