Representing Prompting Patterns with PDL: Compliance Agent Case Study

TL;DR

This paper introduces PDL, a new language for representing and tuning prompts in large language models, demonstrated through a compliance agent case study that significantly improved performance.

Contribution

The paper presents PDL, a novel declarative prompt representation framework that enhances customization and optimization of prompts for LLMs, addressing limitations of existing methods.

Findings

Up to 4x performance improvement with tuned prompts

PDL enables manual and automatic prompt tuning

Demonstrated utility through a real-world compliance agent case study

Abstract

Prompt engineering for LLMs remains complex, with existing frameworks either hiding complexity behind restrictive APIs or providing inflexible canned patterns that resist customization -- making sophisticated agentic programming challenging. We present the Prompt Declaration Language (PDL), a novel approach to prompt representation that tackles this fundamental complexity by bringing prompts to the forefront, enabling manual and automatic prompt tuning while capturing the composition of LLM calls together with rule-based code and external tools. By abstracting away the plumbing for such compositions, PDL aims at improving programmer productivity while providing a declarative representation that is amenable to optimization. This paper demonstrates PDL's utility through a real-world case study of a compliance agent. Tuning the prompting pattern of this agent yielded up to 4x performance improvement compared to using a canned agent and prompt pattern.