FasterPy: An LLM-based Code Execution Efficiency Optimization Framework

TL;DR

FasterPy is a novel framework leveraging LLMs, RAG, and LoRA to efficiently optimize Python code performance, outperforming existing models on the PIE benchmark with low development costs.

Contribution

This work introduces FasterPy, combining retrieval-augmented generation and low-rank adaptation to improve code optimization using LLMs, with demonstrated superior performance on benchmark datasets.

Findings

Outperforms existing models on the PIE benchmark

Uses RAG and LoRA to enhance code optimization

Provides a low-cost, scalable solution for Python code efficiency

Abstract

Code often suffers from performance bugs. These bugs necessitate the research and practice of code optimization. Traditional rule-based methods rely on manually designing and maintaining rules for specific performance bugs (e.g., redundant loops, repeated computations), making them labor-intensive and limited in applicability. In recent years, machine learning and deep learning-based methods have emerged as promising alternatives by learning optimization heuristics from annotated code corpora and performance measurements. However, these approaches usually depend on specific program representations and meticulously crafted training datasets, making them costly to develop and difficult to scale. With the booming of Large Language Models (LLMs), their remarkable capabilities in code generation have opened new avenues for automated code optimization. In this work, we proposed FasterPy, a low-cost and efficient framework that adapts LLMs to optimize the execution efficiency of Python code. FasterPy combines Retrieval-Augmented Generation (RAG), supported by a knowledge base constructed from existing performance-improving code pairs and corresponding performance measurements, with Low-Rank Adaptation (LoRA) to enhance code optimization performance. Our experimental results on the Performance Improving Code Edits (PIE) benchmark demonstrate that our method outperforms existing models on multiple metrics. The FasterPy tool and the experimental results are available at https://github.com/WuYue22/fasterpy.