Hardless: A Generalized Serverless Compute Architecture for Hardware Processing Accelerators

TL;DR

Hardless is a generalized serverless architecture designed to simplify deployment and management of diverse hardware accelerators in cloud environments, enabling scalable and uniform execution of hardware-accelerated workloads.

Contribution

It introduces an extensible serverless framework that supports various hardware accelerators with a unified programming model, addressing current limitations in cloud-based hardware acceleration.

Findings

Supports multiple hardware accelerators with a common model

Enables scalable deployment of hardware-accelerated workloads

Simplifies development and management of hardware-specific applications

Abstract

The increasing use of hardware processing accelerators tailored for specific applications, such as the Vision Processing Unit (VPU) for image recognition, further increases developers' configuration, development, and management overhead. Developers have successfully used fully automated elastic cloud services such as serverless computing to counter these additional efforts and shorten development cycles for applications running on CPUs. Unfortunately, current cloud solutions do not yet provide these simplifications for applications that require hardware acceleration. However, as the development of specialized hardware acceleration continues to provide performance and cost improvements, it will become increasingly important to enable ease of use in the cloud. In this paper, we present an initial design and implementation of Hardless, an extensible and generalized serverless computing architecture that can support workloads for arbitrary hardware accelerators. We show how Hardless can scale across different commodity hardware accelerators and support a variety of workloads using the same execution and programming model common in serverless computing today.