TAPO: Task-Referenced Adaptation for Prompt Optimization

TL;DR

TAPO is a multitask-aware prompt optimization framework that enhances task-specific prompt generation and evaluation for large language models, improving adaptability and performance across diverse datasets.

Contribution

The paper introduces TAPO, a novel framework with task-aware metrics, multi-metrics evaluation, and evolution-based optimization for automatic prompt refinement.

Findings

Effective across six datasets

Improves task-specific prompt quality

Enhances adaptability of prompts

Abstract

Prompt engineering can significantly improve the performance of large language models (LLMs), with automated prompt optimization (APO) gaining significant attention due to the time-consuming and laborious nature of manual prompt design. However, much of the existing work in APO overlooks task-specific characteristics, resulting in prompts that lack domain specificity and are not well-suited for task-specific optimization. In this paper, we introduce TAPO, a multitask-aware prompt optimization framework composed of three key modules. First, a task-aware metric selection module is proposed to enhance task-specific prompt generation capabilities. Second, we present a multi-metrics evaluation module to jointly evaluate prompts from multiple perspectives. Third, an evolution-based optimization framework is introduced for automatic prompt refinement, which improves adaptability across various tasks. Extensive experiments on six datasets demonstrate the effectiveness of our approach, and our code is publicly available.