PAFT: Preservation Aware Fine-Tuning for Minimal-Edit Program Repair

TL;DR

PAFT is a fine-tuning method for program repair that emphasizes minimal edits by preserving stable code regions, leading to more localized and plausible patches without extra inference steps.

Contribution

It introduces a preservation-aware fine-tuning approach that aligns buggy and fixed code to improve minimal edits in program repair models.

Findings

PAFT improves pass@1 by up to 65.6% over standard fine-tuning.

PAFT reduces average edit distance by up to 32.6%.

PAFT outperforms AdaPatcher on Defects4J with higher repair accuracy.

Abstract

Large language models (LLMs) are effective for automated program repair, but plausible patches that pass the full test suite often rewrite more code than necessary, increasing review and maintenance costs. This over-editing is common because most bugs are localized, while standard supervised fine-tuning provides no explicit signal about which tokens should be preserved and which should be changed. We propose PAFT, a preservation-aware fine-tuning method for minimal-edit program repair. PAFT derives token-level preservation signals by aligning buggy and fixed code, combines them with full-sequence masking, and applies an edit-difficulty curriculum. Across Defects4J and HumanEval-Java, PAFT improves pass@1 by up to 65.6% over standard supervised fine-tuning (StdFT) while reducing average edit distance (AED) by up to 32.6%. On Defects4J with DeepSeek-Coder-6.7B, PAFT also outperforms AdaPatcher, a strong preference-based repair baseline, improving pass@1 from 5.9% to 10.1% while reducing median AED from 61.0 to 42.0. Overall, PAFT preserves stable context and concentrates edits on faulty regions, yielding smaller, more localized, plausible patches without inference-time search, reranking, or post-processing.