DeepPERF: A Deep Learning-Based Approach For Improving Software Performance

TL;DR

DeepPERF leverages transformer-based deep learning to automate performance improvement suggestions for C# applications, achieving promising accuracy and real-world validation through open source contributions.

Contribution

This paper introduces DeepPERF, a novel transformer-based model pretrained on code and text, fine-tuned to generate performance patches for C# applications, advancing automation in performance optimization.

Findings

DeepPERF matches developer fixes in ~53% of cases.

Approximately 34% of suggestions are verbatim developer fixes.

Submitted 19 pull requests with 11 approved by project owners.

Abstract

Improving software performance is an important yet challenging part of the software development cycle. Today, the majority of performance inefficiencies are identified and patched by performance experts. Recent advancements in deep learning approaches and the wide-spread availability of open source data creates a great opportunity to automate the identification and patching of performance problems. In this paper, we present DeepPERF, a transformer-based approach to suggest performance improvements for C# applications. We pretrain DeepPERF on English and Source code corpora and followed by finetuning for the task of generating performance improvement patches for C# applications. Our evaluation shows that our model can generate the same performance improvement suggestion as the developer fix in ~53% of the cases, getting ~34% of them verbatim in our expert-verified dataset of performance changes made by C# developers. Additionally, we evaluate DeepPERF on 50 open source C# repositories on GitHub using both benchmark and unit tests and find that our model is able to suggest valid performance improvements that can improve both CPU usage and Memory allocations. So far we've submitted 19 pull-requests with 28 different performance optimizations and 11 of these PRs have been approved by the project owners.