Hybrid Computing for Interactive Datacenter Applications

TL;DR

This paper introduces Spork, a hybrid FPGA-CPU scheduler that optimizes energy efficiency and cost for datacenter workloads by leveraging FPGAs for stable tasks and CPUs for bursts, demonstrating significant improvements over homogeneous and existing hybrid systems.

Contribution

The paper presents Spork, a novel lightweight hybrid scheduler that dynamically balances FPGA and CPU resources to optimize energy and cost efficiency in datacenter applications.

Findings

Energy-optimized Spork is 1.53x more energy efficient and 2.14x cheaper than FPGA-only platforms.

Spork achieves 1.2-2.4x higher energy efficiency than an idealized hybrid scheduler.

Cost-optimized Spork reduces costs by 1.06-1.2x while maintaining higher energy efficiency.

Abstract

Field-Programmable Gate Arrays (FPGAs) are more energy efficient and cost effective than CPUs for a wide variety of datacenter applications. Yet, for latency-sensitive and bursty workloads, this advantage can be difficult to harness due to high FPGA spin-up costs. We propose that a hybrid FPGA and CPU computing framework can harness the energy efficiency benefits of FPGAs for such workloads at reasonable cost. Our key insight is to use FPGAs for stable-state workload and CPUs for short-term workload bursts. Using this insight, we design Spork, a lightweight hybrid scheduler that can realize these energy efficiency and cost benefits in practice. Depending on the desired objective, Spork can trade off energy efficiency for cost reduction and vice versa. It is parameterized with key differences between FPGAs and CPUs in terms of power draw, performance, cost, and spin-up latency. We vary this parameter space and analyze various application and worker configurations on production and synthetic traces. Our evaluation of cloud workloads shows that energy-optimized Spork is not only more energy efficient but it is also cheaper than homogeneous platforms--for short application requests with tight deadlines, it is 1.53x more energy efficient and 2.14x cheaper than using only FPGAs. Relative to an idealized version of an existing cost-optimized hybrid scheduler, energy-optimized Spork provides 1.2-2.4x higher energy efficiency at comparable cost, while cost-optimized Spork provides 1.1-2x higher energy efficiency at 1.06-1.2x lower cost.