Generation and Application of Constrained Interaction Test Suites Using Base Forbidden Tuples With Mixed Neighborhood Tabu Search

TL;DR

This paper introduces a novel search-based method using base forbidden tuples and mixed neighborhood tabu search to efficiently generate constrained interaction test suites, improving coverage and fault detection in software testing.

Contribution

It proposes a new strategy combining BFT filtering and mixed neighborhood TS for constrained test suite generation, outperforming existing tools on standard benchmarks.

Findings

Outperforms ACTS in 83% of benchmarks for interaction strength 2.

Achieves similar results to CASA for 65% of benchmarks at interaction strength 2.

Successfully generates test suites for interaction strength 4, which many strategies cannot.

Abstract

Nowadays, ensuring the quality becomes challenging for most modern software systems when constraints are given for the combinations of configurations. Combinatorial interaction strategies can systematically reduce the number of test cases to construct a minimal test suite without affecting the effectiveness of the tests. This paper presents a new efficient search-based strategy to generate constrained interaction test suites to cover all possible combinations. The paper also shows a new application of constrained interaction testing in software fault searches. The proposed strategy initially generates the set of all possible t-tuple combinations; then, it filters out the set by removing the forbidden t-tuples using the base forbidden tuple (BFT) approach. The strategy also utilizes a mixed neighborhood tabu search (TS) to construct optimal or near-optimal constrained test suites. The efficiency of the proposed method is evaluated through a comparison against two well-known state-of-the-art tools. The evaluation consists of three sets of experiments for 35 standard benchmarks. Additionally, the effectiveness and quality of the results are assessed using a real-world case study. Experimental results show that the proposed strategy outperforms one of the competitive strategies, ACTS, for approximately 83% of the benchmarks and achieves similar results to CASA for 65% of the benchmarks when the interaction strength is 2. For an interaction strength of 3, the proposed method outperforms other competitive strategies for approximately 60% and 42% of the benchmarks. The proposed strategy can also generate constrained interaction test suites for an interaction strength of 4, which is not possible for many strategies. Real-world case study shows that the generated test suites can effectively detect injected faults using mutation testing.