PEACE: Towards Efficient Project-Level Efficiency Optimization via Hybrid Code Editing

TL;DR

PEACE is a hybrid code editing framework that improves project-level code efficiency by considering function interactions, ensuring correctness, and outperforming existing methods on real-world Python projects.

Contribution

It introduces a novel hybrid framework for project-level code efficiency optimization that accounts for function interactions and correctness, along with a new benchmark dataset.

Findings

Achieves 69.2% correctness rate in code optimization tasks.

Attains a 46.9% improvement in optimization rate.

Realizes a 0.840 speedup in execution efficiency.

Abstract

Large Language Models (LLMs) have demonstrated significant capability in code generation, but their potential in code efficiency optimization remains underexplored. Previous LLM-based code efficiency optimization approaches exclusively focus on function-level optimization and overlook interaction between functions, failing to generalize to real-world development scenarios. Code editing techniques show great potential for conducting project-level optimization, yet they face challenges associated with invalid edits and suboptimal internal functions. To address these gaps, we propose Peace, a novel hybrid framework for Project-level code Efficiency optimization through Automatic Code Editing, which also ensures the overall correctness and integrity of the project. Peace integrates three key phases: dependency-aware optimizing function sequence construction, valid associated edits identification, and efficiency optimization editing iteration. To rigorously evaluate the effectiveness of Peace, we construct PeacExec, the first benchmark comprising 146 real-world optimization tasks from 47 high-impact GitHub Python projects, along with highly qualified test cases and executable environments. Extensive experiments demonstrate Peace's superiority over the state-of-the-art baselines, achieving a 69.2% correctness rate (pass@1), +46.9% opt rate, and 0.840 speedup in execution efficiency. Notably, our Peace outperforms all baselines by significant margins, particularly in complex optimization tasks with multiple functions. Moreover, extensive experiments are also conducted to validate the contributions of each component in Peace, as well as the rationale and effectiveness of our hybrid framework design.