Continuous and Resource Managed Regression Testing: An Industrial Use Case

TL;DR

This paper presents a resource-managed regression testing framework using test case prioritisation, aimed at improving cost-effectiveness and resource allocation in industrial embedded software development.

Contribution

It introduces a functional test case prioritisation method based on version control and regression history for resource-efficient regression testing.

Findings

Selected 0.4%-2.0% of test cases for prioritisation

Enhanced regression testing for agile development

Applicable to industrial embedded systems

Abstract

Regression testing is an important part of quality control in both software and embedded products, where hardware is involved. It is also one of the most expensive and time consuming part of the product cycle. To improve the cost effectiveness of the development cycle and the regression testing, we use test case prioritisation and selection techniques to run more important test cases earlier in the testing process. In this paper, we consider a functional test case prioritisation with an access only to the version control of the codebase and regression history. Prioritisation is used to aid our test case selection, where we have chosen 5-25 (0.4%-2.0% of 1254) test cases to validate our method. The selection technique together with other prioritisation methods allows us to shape the current static, retest-all regression testing into a more resource managed regression testing framework. This framework will serve the agile way of working better and will allow us to allocate testing resources more wisely. This is a joint work with a large international Finnish company in an embedded industrial domain.