EfficientEdit: Accelerating Code Editing via Edit-Oriented Speculative Decoding

TL;DR

EfficientEdit significantly accelerates code editing with a novel speculative decoding approach that reuses code segments and employs high-quality drafts, achieving over 10x speedups while maintaining quality.

Contribution

This paper introduces EfficientEdit, a new method that enhances LLM-based code editing efficiency by leveraging localized changes and a dynamic verification mechanism.

Findings

Achieves up to 13.09× speedup over standard decoding.

Outperforms existing inference acceleration methods by up to 90.6%.

Effectively reuses code segments and high-quality drafts for faster editing.

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in code editing, substantially enhancing software development productivity. However, the inherent complexity of code editing tasks forces existing approaches to rely on LLMs' autoregressive end-to-end generation, where decoding speed plays a critical role in efficiency. While inference acceleration techniques like speculative decoding are applied to improve the decoding efficiency, these methods fail to account for the unique characteristics of code editing tasks where changes are typically localized and existing code segments are reused. To address this limitation, we propose EfficientEdit, a novel method that improves LLM-based code editing efficiency through two key mechanisms based on speculative decoding: (1) effective reuse of original code segments while identifying potential edit locations, and (2) efficient generate edit content via high-quality drafts from edit-oriented draft models and a dynamic verification mechanism that balances quality and acceleration. Experimental results show that EfficientEdit can achieve up to 10.38$\times$ and 13.09$\times$ speedup compared to standard autoregressive decoding in CanItEdit and CodeIF-Bench, respectively, outperforming state-of-the-art inference acceleration approaches by up to 90.6%. The code and data are available at https://github.com/zhu-zhu-ding/EfficientEdit.