Preemption-Enhanced Benchmark Suite for FPGAs

TL;DR

This paper introduces the first open-source benchmark suite for evaluating FPGA preemption strategies, featuring diverse applications and comprehensive context management to enable consistent, reproducible research in FPGA scheduling and preemption.

Contribution

It provides a standardized, extensible benchmarking framework for FPGA preemption and scheduling research, addressing the lack of common evaluation tools in the field.

Findings

Includes 27 diverse FPGA applications for benchmarking.

Facilitates reproducible evaluation of preemption and scheduling.

Supports research on fairness, resource efficiency, and context-switching.

Abstract

Field-Programmable Gate Arrays (FPGAs) have become essential in cloud computing due to their reconfigurability, energy efficiency, and ability to accelerate domain-specific workloads. As FPGA adoption grows, research into task scheduling and preemption techniques has intensified. However, the field lacks a standardized benchmarking framework for consistent and reproducible evaluation. Many existing studies propose innovative scheduling or preemption mechanisms but often rely on proprietary or synthetic benchmarks, limiting generalizability and making comparison difficult. This methodical fragmentation hinders effective evaluation of scheduling strategies and preemption in multi-tenant FPGA environments. This paper presents the first open-source preemption-enabled benchmark suite for evaluating FPGA preemption strategies and testing new scheduling algorithms, without requiring users to create preemption workloads from scratch. The suite includes 27 diverse applications spanning cryptography, AI/ML, computation-intensive workloads, communication systems, and multimedia processing. Each benchmark integrates comprehensive context-saving and restoration mechanisms, facilitating reproducible research and consistent comparisons. Our suite not only simplifies testing FPGA scheduling policies but also benefits OS research by enabling the evaluation of scheduling fairness, resource allocation efficiency, and context-switching performance in multi-tenant FPGA systems, ultimately supporting the development of better operating systems and scheduling policies for FPGA-based environments. We also provide guidelines for adding new benchmarks, enabling future research to expand and refine FPGA preemption and scheduling evaluation.