Prompt Optimization Via Diffusion Language Models

TL;DR

This paper introduces a diffusion-based method for optimizing prompts to improve large language model performance, using iterative, span-level updates without modifying the underlying models.

Contribution

It presents a novel diffusion language model framework for prompt refinement that is flexible, model-agnostic, and does not require gradient access or model fine-tuning.

Findings

Consistent performance improvements across multiple benchmarks.

Moderate diffusion steps balance refinement quality and stability.

Method is scalable and applicable to various LLMs.

Abstract

We propose a diffusion-based framework for prompt optimization that leverages Diffusion Language Models (DLMs) to iteratively refine system prompts through masked denoising. By conditioning on interaction traces, including user queries, model responses, and optional feedback, our method enables flexible, span-level prompt updates without requiring gradient access or modifying the downstream language model. Across diverse benchmarks (e.g., $\tau$-bench, SST-2, SST-5), DLM-optimized prompts consistently improve the performance of a frozen target LLM (e.g., GPT-4o-mini). We further show that moderate diffusion step counts provide the best balance between refinement quality and stability. These results highlight diffusion-based prompt optimization as a general, model-agnostic, and scalable approach for enhancing LLM performance through iterative prompt refinement.