The Westermo test results data set

TL;DR

The paper introduces the Westermo test results data set, a large collection of over one million verdicts from embedded system testing, aimed at supporting research in software testing challenges.

Contribution

It provides a comprehensive, realistic dataset of test verdicts and code changes to aid research in regression testing, flaky tests, and test visualization.

Findings

Over one million test verdicts included

Data spans more than five hundred days of testing

Contains information on code changes and test outcomes

Abstract

There is a growing body of knowledge in the computer science, software engineering, software testing and software test automation disciplines. However, there is a challenge for researchers to evaluate their research findings, innovations and tools due to lack of realistic data. This paper presents the Westermo test results data set, more than one million verdicts from testing of embedded systems, from more than five hundred consecutive days of nightly testing. The data also contains information on code changes in both the software under test and the test framework used for testing. This data set can support the research community in particular with respect to the regression test selection problem, flaky tests, test results visualization, etc.