A Data-driven ML Approach for Maximizing Performance in LLM-Adapter Serving

TL;DR

This paper presents a data-driven machine learning method and a Digital Twin for optimizing GPU throughput in LLM-adapter serving systems, effectively balancing performance and resource constraints.

Contribution

It introduces the first Digital Twin for LLM-adapter serving, enabling efficient training data generation and accurate throughput prediction under heterogeneous workloads.

Findings

Digital Twin reproduces throughput within 5.1% of real results

ML approach predicts optimal adapter configurations with at most 7.2% error

Method improves GPU utilization and prevents request starvation

Abstract

With the rapid adoption of Large Language Models (LLMs), LLM-adapters have become increasingly common, providing lightweight specialization of large-scale models. Serving hundreds or thousands of these adapters on a single GPU allows request aggregation, increasing throughput, but may also cause request starvation if GPU memory limits are exceeded. To address this issue, this study focuses on determining the joint configuration of concurrent and parallel adapters that maximizes GPU throughput without inducing starvation, given heterogeneous adapter and traffic properties. We propose a data-driven ML approach leveraging interpretable models to tackle this caching problem and introduce the first Digital Twin capable of reproducing an LLM-adapter serving system, enabling efficient training data generation. Experiments with the vLLM framework and LoRA adapters show that the Digital Twin reproduces throughput within 5.1% of real results, while the ML approach predicts optimal numbers of concurrent and parallel adapters with an error of at most 7.2% under heterogeneous, real-world workloads. The code is publicly available at https://github.com/FerranAgulloLopez/GPULLMAdapterOptimization.