PAFT: Prompt-Agnostic Fine-Tuning

TL;DR

PAFT is a novel fine-tuning method that improves large language models' robustness to prompt variations by training with diverse synthetic prompts, leading to better generalization and faster inference.

Contribution

This paper introduces PAFT, a prompt-agnostic fine-tuning approach that enhances robustness and performance of LLMs against prompt phrasing variations.

Findings

7% higher accuracy on unseen prompts

Superior performance on question answering, reasoning, and tool use benchmarks

Models trained with PAFT are 3.2 times faster in inference

Abstract

Fine-tuning large language models (LLMs) often causes overfitting to specific prompt wording, where minor phrasing variations drastically reduce performance. To address this, we propose Prompt-Agnostic Fine-Tuning (PAFT), a method that enhances robustness through dynamic prompt variation during training. PAFT first generates diverse synthetic prompts, then continuously samples from this set to construct training instances, forcing models to learn fundamental task principles rather than surface-level patterns. Across systematic evaluations using both supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RLFT), PAFT demonstrates substantially improved prompt robustness, achieving 7% higher generalization accuracy on unseen prompts than standard methods. In addition to enhanced robustness, PAFT consistently yields superior overall performance on established benchmarks for question answering, mathematical reasoning, and tool use. Notably, models trained with PAFT attain 3.2 faster inference speeds due to reduced prompt sensitivity. Ablation studies further validate effectiveness of PAFT, while theoretical analysis reveals that PAFT can effectively enhance the cross-domain generalization ability of LLM.