DIP: Dynamic In-Context Planner For Diffusion Language Models

TL;DR

This paper introduces DIP, a method that dynamically adjusts in-context examples during diffusion language model generation, significantly improving inference speed while maintaining quality.

Contribution

DIP leverages the diffusion paradigm's ability for efficient context adjustment, enabling dynamic in-context example selection during generation.

Findings

Achieves up to 12.9× inference speedup.

Maintains generation quality comparable to standard methods.

Outperforms KV cache-enhanced inference by 1.17×.

Abstract

Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. However, due to the bidirectional attention mechanism, DLMs incur substantial computational cost as context length increases. This work addresses this issue with a key discovery: unlike the sequential generation in autoregressive language models (ARLMs), the diffusion generation paradigm in DLMs allows \textit{efficient dynamic adjustment of the context} during generation. Building on this insight, we propose \textbf{D}ynamic \textbf{I}n-Context \textbf{P}lanner (DIP), a context-optimization method that dynamically selects and inserts in-context examples during generation, rather than providing all examples in the prompt upfront. Results show DIP maintains generation quality while achieving up to 12.9$\times$ inference speedup over standard inference and 1.17$\times$ over KV cache-enhanced inference.