UltraShare: FPGA-based Dynamic Accelerator Sharing and Allocation

TL;DR

UltraShare is a hardware and software framework that enables dynamic, non-blocking sharing of FPGA accelerators among multiple applications, significantly improving throughput and resource utilization.

Contribution

It introduces UltraShare, a scalable hardware controller with a supporting software stack that supports dynamic, congestion-free sharing of multiple FPGA accelerators among applications.

Findings

Up to 8x throughput improvement in accelerator utilization

Supports dynamic parallelism and multi-application sharing

Provides a non-blocking, congestion-free environment for FPGA accelerators

Abstract

Despite all the available commercial and open-source frameworks to ease deploying FPGAs in accelerating applications, the current schemes fail to support sharing multiple accelerators among various applications. There are three main features that an accelerator sharing scheme requires to support: exploiting dynamic parallelism of multiple accelerators for a single application, sharing accelerators among multiple applications, and providing a non-blocking congestion-free environment for applications to invoke the accelerators. In this paper, we developed a scalable fully functional hardware controller, called UltraShare, with a supporting software stack that provides a dynamic accelerator sharing scheme through an accelerators grouping mechanism. UltraShare allows software applications to fully utilize FPGA accelerators in a non-blocking congestion-free environment. Our experimental results for a simple scenario of a combination of three streaming accelerators invocation show an improvement of up to 8x in throughput of the accelerators by removing accelerators idle times.