PRL: Prompts from Reinforcement Learning

TL;DR

PRL introduces a reinforcement learning-based method for automatic prompt generation that outperforms existing approaches across multiple NLP benchmarks, reducing reliance on expert-crafted prompts.

Contribution

The paper presents a novel RL-based approach for generating effective prompts, capable of producing unseen few-shot examples and achieving state-of-the-art results.

Findings

Surpasses prior methods by 2.58% on classification accuracy

Improves ROUGE scores by 4.32 on summarization

Enhances SARI scores by 6.93 on simplification

Abstract

Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting them typically demands expert intuition and a nuanced understanding of the task. Moreover, the most impactful prompts often hinge on subtle semantic cues, ones that may elude human perception but are crucial for guiding LLM behavior. In this paper, we introduce PRL (Prompts from Reinforcement Learning), a novel RL-based approach for automatic prompt generation. Unlike previous methods, PRL can produce novel few-shot examples that were not seen during training. Our approach achieves state-of-the-art performance across a range of benchmarks, including text classification, simplification, and summarization. On the classification task, it surpasses prior methods by 2.58% over APE and 1.00% over EvoPrompt. Additionally, it improves the average ROUGE scores on the summarization task by 4.32 over APE and by 2.12 over EvoPrompt and the SARI score on simplification by 6.93 over APE and by 6.01 over EvoPrompt. Our code is available at https://github.com/Batorskq/prl .