TEP-GNN: Accurate Execution Time Prediction of Functional Tests using Graph Neural Networks

TL;DR

TEP-GNN introduces a graph neural network-based approach using flow-augmented ASTs to accurately predict unit test execution times in Java projects, showing promising results but needing further generalization improvements.

Contribution

The paper presents TEP-GNN, a novel GNN model utilizing FA-ASTs for high-accuracy performance prediction of unit tests, advancing code performance forecasting methods.

Findings

Achieves a Pearson correlation of 0.789 in predictions.

Outperforms baseline deep learning models.

Highlights the need for better generalization to unseen projects.

Abstract

Predicting the performance of production code prior to actually executing or benchmarking it is known to be highly challenging. In this paper, we propose a predictive model, dubbed TEP-GNN, which demonstrates that high-accuracy performance prediction is possible for the special case of predicting unit test execution times. TEP-GNN uses FA-ASTs, or flow-augmented ASTs, as a graph-based code representation approach, and predicts test execution times using a powerful graph neural network (GNN) deep learning model. We evaluate TEP-GNN using four real-life Java open source programs, based on 922 test files mined from the projects' public repositories. We find that our approach achieves a high Pearson correlation of 0.789, considerable outperforming a baseline deep learning model. However, we also find that more work is needed for trained models to generalize to unseen projects. Our work demonstrates that FA-ASTs and GNNs are a feasible approach for predicting absolute performance values, and serves as an important intermediary step towards being able to predict the performance of arbitrary code prior to execution.