Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines

TL;DR

Llama is a novel serverless framework that automatically optimizes video analytics pipelines for cost and latency efficiency by dynamically assigning configurations across heterogeneous hardware, handling input variability and complex DAGs.

Contribution

Llama introduces an auto-tuning framework for heterogeneous, serverless video pipelines that dynamically optimizes configurations based on latency targets and resource costs.

Findings

Llama achieves 7.8x lower latency than existing systems.

Llama reduces costs by 16x on average.

It effectively handles input-dependent behavior and complex DAGs.

Abstract

The proliferation of camera-enabled devices and large video repositories has led to a diverse set of video analytics applications. These applications rely on video pipelines, represented as DAGs of operations, to transform videos, process extracted metadata, and answer questions like, "Is this intersection congested?" The latency and resource efficiency of pipelines can be optimized using configurable knobs for each operation (e.g., sampling rate, batch size, or type of hardware used). However, determining efficient configurations is challenging because (a) the configuration search space is exponentially large, and (b) the optimal configuration depends on users' desired latency and cost targets, (c) input video contents may exercise different paths in the DAG and produce a variable amount intermediate results. Existing video analytics and processing systems leave it to the users to manually configure operations and select hardware resources. We present Llama: a heterogeneous and serverless framework for auto-tuning video pipelines. Given an end-to-end latency target, Llama optimizes for cost efficiency by (a) calculating a latency target for each operation invocation, and (b) dynamically running a cost-based optimizer to assign configurations across heterogeneous hardware that best meet the calculated per-invocation latency target. This makes the problem of auto-tuning large video pipelines tractable and allows us to handle input-dependent behavior, conditional branches in the DAG, and execution variability. We describe the algorithms in Llama and evaluate it on a cloud platform using serverless CPU and GPU resources. We show that compared to state-of-the-art cluster and serverless video analytics and processing systems, Llama achieves 7.8x lower latency and 16x cost reduction on average.