$\lambda$-NIC: Interactive Serverless Compute on Programmable SmartNICs

TL;DR

This paper introduces $\lambda$-NIC, a framework that enables running interactive serverless workloads directly on SmartNICs, achieving significant latency and throughput improvements over traditional server CPU-based approaches.

Contribution

It presents a novel framework for executing serverless functions on SmartNICs, leveraging their proximity to the network and multiple cores for high concurrency and low latency.

Findings

Achieves up to 880x latency reduction

Achieves up to 736x throughput increase

Reduces host CPU and memory usage significantly

Abstract

There is a growing interest in serverless compute, a cloud computing model that automates infrastructure resource-allocation and management while billing customers only for the resources they use. Workloads like stream processing benefit from high elasticity and fine-grain pricing of these serverless frameworks. However, so far, limited concurrency and high latency of server CPUs prohibit many interactive workloads (e.g., web servers and database clients) from taking advantage of serverless compute to achieve high performance. In this paper, we argue that server CPUs are ill-suited to run serverless workloads (i.e., lambdas) and present $\lambda$-NIC, an open-source framework, that runs interactive workloads directly on a SmartNIC; more specifically an ASIC-based NIC that consists of a dense grid of Network Processing Unit (NPU) cores. $\lambda$-NIC leverages SmartNIC's proximity to the network and a vast array of NPU cores to simultaneously run thousands of lambdas on a single NIC with strict tail-latency guarantees. To ease development and deployment of lambdas, $\lambda$-NIC exposes an event-based programming abstraction, Match+Lambda, and a machine model that allows developers to compose and execute lambdas on SmartNICs easily. Our evaluation shows that $\lambda$-NIC achieves up to 880x and 736x improvements in workloads' response latency and throughput, respectively, while significantly reducing host CPU and memory usage.