Making Prompts First-Class Citizens for Adaptive LLM Pipelines

TL;DR

This paper introduces SPEAR, a novel framework that treats prompts as first-class citizens in LLM pipelines, enabling structured management, adaptive refinement, and policy-driven control for improved flexibility and reuse.

Contribution

SPEAR's design allows prompts to be dynamically refined during execution, enhancing adaptability and integration within complex LLM workflows.

Findings

Preliminary results show promising potential for runtime prompt refinement.

Structured prompt management improves introspection and provenance tracking.

Policy-driven control enables automatic prompt updates based on runtime feedback.

Abstract

Modern LLM pipelines increasingly resemble complex data-centric applications: they retrieve data, correct errors, call external tools, and coordinate interactions between agents. Yet, the central element controlling this entire process -- the prompt -- remains a brittle, opaque string that is entirely disconnected from the surrounding program logic. This disconnect fundamentally limits opportunities for reuse, optimization, and runtime adaptivity. In this paper, we describe our vision and an initial design of SPEAR (Structured Prompt Execution and Adaptive Refinement), a new approach to prompt management that treats prompts as first-class citizens in the execution model. Specifically, SPEAR enables: (1) structured prompt management, with prompts organized into versioned views to support introspection and reasoning about provenance; (2) adaptive prompt refinement, whereby prompts can evolve dynamically during execution based on runtime feedback; and (3) policy-driven control, a mechanism for the specification of automatic prompt refinement logic as when-then rules. By tackling the problem of runtime prompt refinement, SPEAR plays a complementary role in the vast ecosystem of existing prompt optimization frameworks and semantic query processing engines. We describe a number of related optimization opportunities unlocked by the SPEAR model, and our preliminary results demonstrate the strong potential of this approach.