Auto-Prompt Generation is Not Robust: Prompt Optimization Driven by Pseudo Gradient

TL;DR

This paper introduces PertBench, a benchmark for evaluating prompt robustness, and proposes PGO, a gradient-free method that improves prompt stability against input perturbations in large language models.

Contribution

The paper presents PertBench for systematic robustness evaluation and introduces PGO, a novel gradient-free prompt optimization method that enhances prompt robustness under noisy conditions.

Findings

PGO outperforms existing methods in robustness tests.

PertBench reveals significant vulnerabilities in current prompt strategies.

PGO maintains performance across diverse tasks and models.

Abstract

While automatic prompt generation methods have recently received significant attention, their robustness remains poorly understood. In this paper, we introduce PertBench, a comprehensive benchmark dataset that includes a wide range of input perturbations, designed to systematically evaluate the robustness of current auto-prompting techniques. Our analysis reveals substantial vulnerabilities in existing prompt generation strategies, where even minor modifications to the prompt can lead to significant differences in model output. To address this issue, we propose PGO, a gradient-free prompt generation framework that leverages perturbation types as pseudo-gradient signals to guide LLMs in producing more robust prompts. In contrast to existing methods that assess prompt quality only on clean, well-structured inputs, our approach explicitly emphasizes robustness under noisy and perturbed conditions. Extensive experiments across diverse tasks and multiple LLMs show PGO consistently outperforms previous methods in maintaining performance under input perturbations.