ProRefine: Inference-Time Prompt Refinement with Textual Feedback

TL;DR

ProRefine is a novel inference-time prompt refinement method that uses textual feedback from LLMs to improve multi-step reasoning performance without additional training, significantly surpassing zero-shot baselines.

Contribution

It introduces an inference-time prompt optimization technique using an agentic loop of LLMs, enabling dynamic prompt refinement without extra training or labels.

Findings

Outperforms zero-shot Chain-of-Thought baselines by 3-37 percentage points.

Enables smaller models to approach larger model performance.

Improves accuracy and cost-effectiveness of multi-step reasoning tasks.

Abstract

Agentic workflows, where multiple AI agents collaborate to accomplish complex tasks like reasoning or planning, play a substantial role in many cutting-edge commercial applications, and continue to fascinate researchers across fields for their potential to accomplish expensive, complex tasks that, until recently, only humans have been trusted to do. These workflows depend critically on the prompts used to provide the roles models play in such workflows. Poorly designed prompts that fail even slightly to guide individual agents can lead to sub-optimal performance that may snowball within a system of agents, limiting their reliability and scalability. To address this important problem of inference-time prompt optimization, we introduce ProRefine, an innovative inference-time optimization method that uses an agentic loop of LLMs to generate and apply textual feedback. ProRefine dynamically refines prompts for multi-step reasoning tasks without additional training or ground truth labels. Evaluated on five benchmark mathematical reasoning datasets, ProRefine significantly surpasses zero-shot Chain-of-Thought baselines by 3 to 37 percentage points. This approach not only boosts accuracy but also allows smaller models to approach the performance of their larger counterparts. This highlights its potential for building more cost-effective and powerful hybrid AI systems, thereby democratizing access to high-performing AI.