Efficient LLM-based Advertising via Model Compression and Parallel Verification

TL;DR

This paper introduces an efficient framework for deploying large language models in advertising by combining model compression techniques and parallel verification to reduce latency and computational costs.

Contribution

It presents a novel framework integrating adaptive group quantization, hierarchical sparsification, and parallel verification for faster LLM inference in advertising.

Findings

Achieves significant speedup in LLM inference for advertising tasks.

Maintains acceptable quality levels despite compression and acceleration.

Demonstrates effectiveness on real-world advertising scenarios.

Abstract

Large language models (LLMs) have shown remarkable potential in advertising scenarios such as ad creative generation and targeted advertising. However, deploying LLMs in real-time advertising systems poses significant challenges due to their high inference latency and computational cost. In this paper, we propose an Efficient Generative Targeting framework that integrates adaptive group quantization, layer-adaptive hierarchical sparsification, and prefix-tree parallel verification to accelerate LLM inference while preserving generation quality. Extensive experiments on two real-world advertising scenarios demonstrate that our framework achieves significant speedup with acceptable quality degradation, making it operationally viable for practical deployments.