To test, or not to test: A proactive approach for deciding complete performance test initiation

TL;DR

This paper introduces a clustering-based method to minimize test inputs and a proactive approach to decide when to initiate complete performance testing, especially useful for large-scale, data-growing environments like CERN.

Contribution

It presents a novel clustering technique for test input reduction and a decision-making approach for performance test initiation in large, evolving data contexts.

Findings

Effective test input minimization using clustering.

Accurate recommendations for test initiation in CERN web services.

Validated approach on real-world code updates.

Abstract

Software performance testing requires a set of inputs that exercise different sections of the code to identify performance issues. However, running tests on a large set of inputs can be a very time-consuming process. It is even more problematic when test inputs are constantly growing, which is the case with a large-scale scientific organization such as CERN where the process of performing scientific experiment generates plethora of data that is analyzed by physicists leading to new scientific discoveries. Therefore, in this article, we present a test input minimization approach based on a clustering technique to handle the issue of testing on growing data. Furthermore, we use clustering information to propose an approach that recommends the tester to decide when to run the complete test suite for performance testing. To demonstrate the efficacy of our approach, we applied it to two different code updates of a web service which is used at CERN and we found that the recommendation for performance test initiation made by our approach for an update with bottleneck is valid.