Learning from Contrastive Prompts: Automated Optimization and Adaptation

TL;DR

The paper introduces Learning from Contrastive Prompts (LCP), a framework that improves prompt optimization and adaptation for large language models using contrastive learning, outperforming existing methods and enhancing cross-model and language robustness.

Contribution

LCP is a novel contrastive learning framework that automates prompt optimization and adaptation, addressing limitations of previous methods that rely only on incorrect samples.

Findings

LCP achieves over 76% win rate in prompt optimization on Big-Bench Hard.

LCP demonstrates strong adaptability across different model versions, families, and languages.

LCP reduces manual effort in prompt engineering for diverse LLM deployment.

Abstract

As LLMs evolve, significant effort is spent on manually crafting prompts. While existing prompt optimization methods automate this process, they rely solely on learning from incorrect samples, leading to a sub-optimal performance. Additionally, an unexplored challenge in the literature is prompts effective for prior models may not perform well on newer versions or different languages. We propose the Learning from Contrastive Prompts (LCP) framework to address these gaps, enhancing both prompt optimization and adaptation. LCP employs contrastive learning to generate effective prompts by analyzing patterns in good and bad prompt examples. Our evaluation on the Big-Bench Hard dataset shows that LCP has a win rate of over 76% over existing methods in prompt optimization and demonstrates strong adaptability across different model versions, families, and languages. LCP offers a systematic approach to prompt engineering, reducing manual effort in deploying LLMs across varied contexts.