Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation

TL;DR

The paper introduces PAD-Rec, a position-aware drafting module that enhances speculative decoding in LLM-based list-wise recommendation, significantly improving inference speed while maintaining quality.

Contribution

It proposes a lightweight, trainable module that encodes item position and draft step information to better model uncertainty and structural cues in generative recommendation.

Findings

Achieves up to 3.1x speedup in inference time.

Provides about 5% average speedup over strong baselines.

Largely preserves recommendation quality despite acceleration.

Abstract

Large language model (LLM)-based generative list-wise recommendation has advanced rapidly, but decoding remains sequential and thus latency-prone. To accelerate inference without changing the target distribution, speculative decoding (SD) uses a small draft model to propose several next tokens at once and a target LLM to verify and accept the longest prefix, skipping multiple steps per round. In generative recommendation, however, each item is represented by multiple semantic-ID tokens, often with separators, and current drafts typically treat these tokens uniformly. This overlooks two practical facts: (i) a token's semantics depend on its within-item slot, and (ii) uncertainty tends to increase with speculation depth. Without modeling these effects, SD's speedups can be limited. We introduce PAD-Rec, Position-Aware Drafting for generative Recommendation, a lightweight module that augments the draft model with two complementary signals. Item position embeddings explicitly encode the within-item slot of each token, strengthening structural awareness. Step position embeddings encode the draft step, allowing the model to adapt to depth-dependent uncertainty and improve proposal quality. To harmonize these signals with base features, we add simple gates: a learnable coefficient for item slots and a context-driven gate for draft steps. The module is trainable, easy to integrate with standard draft models, and adds negligible inference overhead. Extensive experiments on four real-world datasets show up to 3.1x wall-clock speedup and about 5% average wall-clock speedup gain over strong SD baselines, while largely preserving recommendation quality.