Two-stage algorithms for covering array construction

TL;DR

This paper introduces a two-stage framework with algorithms for efficiently constructing covering arrays, significantly reducing the number of tests needed for software component interaction testing.

Contribution

The paper develops a novel two-stage construction framework that combines randomized and sophisticated search algorithms to improve covering array efficiency.

Findings

Outperforms existing methods for 20-60 components with 3-6 options.

Achieves over 50% reduction in the number of tests in some cases.

Extends applicability of algorithms to larger component sets without increasing tests.

Abstract

Modern software systems often consist of many different components, each with a number of options. Although unit tests may reveal faulty options for individual components, functionally correct components may interact in unforeseen ways to cause a fault. Covering arrays are used to test for interactions among components systematically. A two-stage framework, providing a number of concrete algorithms, is developed for the efficient construction of covering arrays. %Our framework divides the construction in two stages. In the first stage, a time and memory efficient randomized algorithm covers most of the interactions. In the second stage, a more sophisticated search covers the remainder in relatively few tests. In this way, the storage limitations of the sophisticated search algorithms are avoided; hence the range of the number of components for which the algorithm can be applied is extended, without increasing the number of tests. Many of the framework instantiations can be tuned to optimize a memory-quality trade-off, so that fewer tests can be achieved using more memory. The algorithms developed outperform the currently best known methods when the number of components ranges from 20 to 60, the number of options for each ranges from 3 to 6, and $t$-way interactions are covered for $t\in \{5,6\}$. In some cases a reduction in the number of tests by more than $50\%$ is achieved.