Tessera: Unlocking Heterogeneous GPUs through Kernel-Granularity Disaggregation

TL;DR

Tessera is a novel kernel disaggregation system that enhances performance and cost efficiency for large model inference on heterogeneous GPUs by aligning kernels with hardware capabilities.

Contribution

It introduces the first kernel-level disaggregation approach that adapts to diverse resource demands within applications, improving efficiency over existing coarse-grained methods.

Findings

Up to 2.3x increase in serving throughput.

Up to 1.6x improvement in cost efficiency.

Heterogeneous GPU pairs can outperform homogeneous high-end GPU setups.

Abstract

Disaggregation maps parts of an AI workload to different types of GPUs, offering a path to utilize modern heterogeneous GPU clusters. However, existing solutions operate at a coarse granularity and are tightly coupled to specific model architectures, leaving much room for performance improvement. This paper presents Tessera, the first kernel disaggregation system to improve performance and cost efficiency on heterogeneous GPUs for large model inference. Our key insight is that kernels within a single application exhibit diverse resource demands, making them the most suitable granularity for aligning computation with hardware capabilities. Tessera integrates offline analysis with online adaptation by extracting precise inter-kernel dependencies from PTX to ensure correctness, overlapping communication with computation through a pipelined execution model, and employing workload-aware scheduling with lightweight runtime adaptation. Extensive evaluations across five heterogeneous GPUs and four model architectures, scaling up to 16 GPUs, show that Tessera improves serving throughput and cost efficiency by up to 2.3x and 1.6x, respectively, compared to existing disaggregation methods, while generalizing to model architectures where prior approaches do not apply. Surprisingly, a heterogeneous GPU pair under Tessera can even exceed the throughput of two homogeneous high-end GPUs at a lower cost.